Collaborative data curation for global food security

Food security decisions depend on more than a single indicator. A useful evidence base combines crop yields, food balance sheets, market prices, nutrition measures, climate observations and humanitarian logistics. These collections often use different identifiers, units and collection methods, so a striking chart can conceal uncertainty or make two regions look comparable when they are not.

At IASSIST 2017, held at the University of Kansas in Lawrence from 23 to 26 May under the theme “Data in the Middle: The Common Language of Research,” speakers addressed big data, deep learning, digital agriculture and global food security. The archived program, schedule, plenaries and presentations show why a common language matters: research becomes useful across sectors when its data can be found, interpreted and responsibly reused.

Why food security data needs shared stewardship

Food security has four connected dimensions: availability, access, utilisation and stability. Collaborative data curation for global food security datasets must keep those dimensions distinct while connecting them. A production total can improve food availability, yet say little about whether households can afford nutritious food. A reliable collection preserves the context needed to interpret each measure.

Relevant records include farm-level yields, livestock inventories, household consumption surveys, food prices and nutrition indicators. Each source has strengths and blind spots. Satellite data may cover broad areas but classify mixed farming systems poorly, while surveys capture lived conditions but cover fewer locations. Data stewardship joins these sources through documentation, agreed definitions and traceable transformations.

Curation is therefore more than tidying files. It involves selection, annotation, metadata, quality control, preservation and access decisions made with the people represented by the data. Agricultural agencies, researchers, humanitarian organisations, farmers and community groups may hold complementary knowledge. Shared stewardship lets them retain authority over sensitive information while making the wider evidence base more useful and trustworthy.

Comparing ways to curate shared datasets

A global collection can be organised through several approaches, and each places a different burden on participants. The appropriate choice depends on governance, technical capacity, expected reuse and the sensitivity of the information. Central repositories, catalogues, commons and community-governed collections can each form part of a sustainable arrangement.

Curation model	What it does well	Where it strains	Best fit
Central repository	Provides one authoritative location and clear version control	Can duplicate existing data and restrict distributed contribution	Well-bounded collections with a stable custodian
Federated catalogue	Connects records while allowing institutions to retain control	Depends on consistent identifiers and active maintenance across sites	Cross-agency and multi-country programmes
Domain data commons	Provides shared standards, vocabularies and reusable components	Requires agreement among communities with different priorities	Repeated analysis of crop, climate and food-system data
Community-governed collection	Keeps sensitive information under the authority of the people represented	May limit openness and require long-term stewardship capacity	Indigenous, local and participant-owned data

These models can overlap. A public catalogue can point to institutional repositories, while community-controlled access governs records about particular places or groups. The key test is whether users can discover the evidence, understand its limitations and follow its history without bypassing legitimate restrictions. Interoperability is valuable when it supports participation rather than forcing every contributor into an identical workflow.

Metadata, provenance and governance

Interoperable data begins with a defined purpose, responsible custodian and documented collection method. Useful metadata identifies the geography, time period, population and units represented by each observation. It also records missing values, sampling decisions and known limitations. These details allow an analyst to distinguish a measured zero from an unreported value and to judge whether two indicators belong in the same model.

Machine-actionable metadata improves on free-text descriptions through data dictionaries, controlled vocabularies and persistent identifiers. Crop names, administrative areas and dates need shared references across institutions. Provenance records then explain where each value came from, which transformations occurred and who approved them. Version histories are especially important for food balance sheets and crop estimates, because later releases may correct earlier assumptions.

FAIR principles—findable, accessible, interoperable and reusable—provide a useful baseline, but openness cannot override ethical obligations. CARE principles for Indigenous data governance add collective benefit, authority to control and responsibility. Sensitive nutrition, health or household-location records may require tiered access, de-identification or data-use agreements. Governance should be documented alongside technical metadata so that access decisions remain transparent and reviewable.

Australian conditions that shape data practice

Australian food-system research spans CSIRO, the Australian Bureau of Agricultural and Resource Economics and Sciences, the Bureau of Meteorology, state departments, universities, producer groups and aid agencies. This mixed institutional environment is practical rather than theoretical: a grain grower may supply information to a processor, an exporter and a state drought program, each under different agreements. Aligning these records supports both domestic analysis and export-market reporting.

Distance and intermittent connectivity strongly influence field collection across Australia. Producers in the Darling Downs, the New South Wales wheat belt and remote parts of northern Australia may work with intermittent mobile coverage. Offline capture, delayed transmission and local validation can prevent lost observations. Edge processing also supports real-time warnings without requiring constant connectivity, although it creates additional records that must be reconciled.

Australian English, metric measurements and date conventions can cause subtle interoperability errors. International partners accustomed to month-first written dates may misinterpret day-first values, while UTC offsets differ across the country and daylight-saving practices vary by state. Local terms such as “dryland” and “irrigated” also carry practical meanings that generic crop classifications may omit. Shared metadata should record these conventions rather than assume that apparently similar labels are equivalent.

Aboriginal and Torres Strait Islander data sovereignty adds another essential dimension to food-security research. Data concerning Indigenous-managed landscapes, culturally significant foods and community wellbeing should not be treated as freely extractable assets. Governance arrangements must recognise community authority, consent and benefit, including the right to restrict secondary use. Collaborative curation provides a way to document those conditions and make respect for them visible in the resulting evidence.

Learning through collaborative workshop methods

The IASSIST 2017 program placed shared language at the centre of cross-sector research. Participants could revisit the hands-on workshop sessions to see how collaborative exercises reveal different assumptions before a data project becomes fixed. Workshops also provide space for researchers, practitioners and community representatives to test whether terms such as “household,” “food shortage” and “yield” mean the same thing across organisations.

Useful sessions produce artefacts rather than general agreement alone. Participants might map a shared data flow, identify missing metadata fields, compare classification schemes or test a sample record against another organisation’s requirements. These activities expose practical problems early: duplicated collection, inconsistent denominators, unclear licensing and relationships that a technical discussion could otherwise overlook.

The conference archive also includes the schedule, plenary descriptions, presentations and practical information about check-in, transportation and local recommendations in Lawrence. That logistical material reflects a broader principle behind collaborative data work: exchange depends on clear arrangements for participation. Time zones, language, compensation, access needs and decision rights need attention alongside file formats and analytical methods.

Big data, deep learning and digital agriculture

Large-scale agricultural data brings satellite imagery, weather records, sensor readings, machinery logs and market transactions into the same analytical environment. Curators must check calibration, spatial resolution, temporal alignment and coverage before these sources are combined. A cloud-derived soil moisture estimate and a rain-gauge measurement may describe the same field while representing different quantities, so a combined index requires a defensible relationship between them.

Deep learning can improve crop mapping, yield estimation and early detection of pests or drought stress, yet its training data need especially careful stewardship. Label provenance determines whether an image records confirmed disease, a field worker’s visual assessment or an algorithmic prediction. Spatial autocorrelation can place nearly identical observations in both training and testing sets, inflating reported performance. Curated baselines, regional validation sets and documented error rates make such claims more credible.

Digital agriculture also creates continuous operational data from tractors, irrigation systems and livestock equipment. Australian producers often combine these tools with on-farm records and regional advisory services, making local context essential for interpretation. Model outputs should communicate uncertainty, identify groups or regions where performance is weaker and avoid presenting a forecast as an observation. The model card, source documentation and curation history remain part of the evidence, not administrative extras.

Practical stewardship choices for project teams

Before collection begins, project teams can assign responsibilities that prevent ambiguous decisions:

• Name a data steward for each participating organisation and identify who approves release.
• Define the decisions the dataset will support, its intended users and its geographic scope.
• Agree on shared definitions, units, identifiers and formats before data transfer.
• Document consent, access tiers and community governance for sensitive information.

Before publication and subsequent reuse, teams can establish a clearer release process:

• Record provenance, processing steps, software versions and quality checks for every field.
• Run automated checks for duplicates, invalid ranges, missing values and inconsistent categories.
• Publish a version number, change history, licence and known limitations with the release.
• Assign a maintenance owner and a review date for datasets affected by seasonal or policy changes.

These measures make collaboration durable because they distribute stewardship rather than concentrating it in one technical team. They also help international partners recognise where Australian practices differ, such as day-first dates, the use of metric units or the separation of dryland and irrigated production. For global food security datasets, curation is the connective tissue between evidence and action. A trustworthy collection allows researchers, humanitarian workers, policymakers and communities to interpret the same information, challenge its limitations and build on it together.