Open data for digital agriculture and food security

Digital agriculture turns sensors, satellite imagery, machinery records, weather forecasts and market feeds into decisions that can improve yields, reduce waste and strengthen resilience. Its promise depends on data that can move between farms, research organisations, government agencies, processors and communities. Open data provides that common foundation by making public information discoverable, downloadable and reusable under clear conditions.

The archived IASSIST 2017 conference, “Data in the Middle: The Common Language of Research,” held at the University of Kansas in Lawrence from 23 to 26 May 2017, gathered work on big data, deep learning, digital agriculture and global food security. That pairing remains useful for an Australian audience: sophisticated models have little value when observations, definitions and forecasts remain locked in separate systems.

Why open data belongs at the centre of food security

Food security rests on four connected elements: enough food is available, people can access it, it supports nutritional needs, and supply remains stable through shocks. Each element depends on timely information, yet the data behind food systems is often fragmented. A grain grower may know rainfall and soil moisture from one source, while a processor sees storage and freight conditions through another.

Open data means information that is accessible for use and reuse, with licences, metadata and technical formats that allow people to inspect and combine it. Weather records, satellite-derived crop indicators, pest surveillance, water allocations, price series and food-bank inventories can all contribute. The important distinction is the “open” licence and accessible pathway: a report hidden behind a request form still creates friction.

Farm and commercial data require careful treatment, so openness has limits. Soil maps, public research and government monitoring can often be shared, while proprietary yield files, customer lists and some farm records need access controls. A credible policy distinguishes data that should remain public from data requiring consent, aggregation or commercial confidentiality. This protects legitimate interests while preserving a broad pool of shared knowledge.

From conference themes to everyday farm decisions

The phrase “Data in the Middle” captures a practical requirement in agricultural systems: researchers, extension officers, technology firms, farmers and public planners need a common language for observations and uncertainty. The IASSIST program placed big data and deep learning beside digital agriculture and global food security, showing that technical scale and everyday decisions belong in the same conversation. Its archived schedule and plenary material remain a useful record of that debate.

Deep learning can identify disease from leaf images, predict pasture growth or interpret irregular sensor readings. Such tools need representative training data, accurate labels and clear measures of confidence. If a model learns from one region’s soils, varieties or seasons and is applied elsewhere without local validation, it can produce confident but misleading advice. Open datasets allow researchers to test where a method works and where it fails.

Shared data also shortens the path from observation to action. A farmer’s field reading can be checked against a public rainfall record; an extension alert can include a machine-readable pest report; a policy model can use consistent regional boundaries. Interoperable standards, identifiers and metadata turn these small exchanges into a usable evidence base. Without them, “big data” may simply mean many files that cannot speak to one another.

Australian conditions make shared information practical

Australian conditions make the case for open agricultural information unusually concrete. In the Murray–Darling Basin, drought declarations, irrigation allocations, groundwater levels and water trades shape production across New South Wales, Victoria and South Australia. A transparent record of rainfall, soil moisture and allocation conditions can help growers compare options, while public modelling can show how restrictions affect regional food and fibre markets.

The winter cereal belt of New South Wales and Victoria illustrates the value of connecting farm machinery to public information. Headers, spray logs, yield maps and contract records already exist in many businesses, but ownership and formats vary. Open benchmark data lets a standout crop be compared with district and seasonal patterns without exposing a farmer’s commercial records. Australian farm conversations often revolve around the season, the header, the tank and the cost of getting a crop off; accessible data can inform each of those decisions.

Northern Australia adds a different market chain. Queensland cattle producers manage cattle numbers, pasture condition, transport routes and market prices, often within a beef industry strongly connected to live export and domestic processing. Open pasture, rainfall, animal-health and market information can reduce gaps at the station and in regional towns. At the other end of the chain, Woolworths and Coles hold substantial influence over retail purchasing, while independent food rescues distribute surplus food through charities and community programs. Data can connect production decisions with what retailers and households actually consume.

Designing an open agricultural data commons

A useful data commons needs more than uploads. Core government, research and industry datasets should be discoverable through catalogues, carry consistent metadata, state provenance and freshness, and provide accessible application programming interfaces or bulk downloads. Stable identifiers for regions, farms, crops, pests and diseases prevent the same concept from acquiring several meanings. Versioning matters too, because a model trained on one release should remain traceable to the data that produced it.

Trust depends on governance as much as technical format. Public authorities can clarify which datasets are authoritative and how corrections are issued. Researchers can document sampling limits, model performance and ethical approval. Farmers, small businesses and communities need a role in deciding how information about their operations is collected and shared. Indigenous knowledge and community observations require respect for consent, authority and benefit sharing, rather than automatic digitisation.

Long-term funding is the quiet foundation of open data. A portal with stale feeds can be less useful than a smaller collection maintained regularly. Australian agencies, including the Bureau of Meteorology, the Australian Bureau of Agricultural and Resource Economics and Sciences and state agriculture departments, already produce significant information; the challenge is joining it through shared standards without copying the same file into another portal. Connectivity also varies across remote properties, so published data should remain useful through low-bandwidth channels and plain-language summaries.

Turning shared data into resilient food systems

At farm level, open information supports decisions that are made before a season becomes visible in final yields. Rainfall and temperature records can improve planting windows; soil and water data can inform irrigation; satellite time series can reveal uneven growth; market information can improve timing and storage decisions. These services become more valuable when they combine trusted public data with confidential farm records, rather than pretending that public information alone describes every paddock.

Along the supply chain, shared data can reduce food loss and price uncertainty. Machine-readable reports on storage, transport, inventory and quality can help processors plan capacity and redirect surplus before it spoils. Public price and production series can also help small buyers understand market conditions that are usually visible mainly to large traders. For emergency planning, timely maps of heat, flood or bushfire impacts can support logistics and household access to food, provided the information reaches community organisations in usable forms.

Food security also depends on affordability and equitable distribution. National averages can conceal pressure in regional towns, low-income suburbs and remote communities. Aggregated food-bank and school-meal data, together with local price monitoring, can identify areas where support is needed, while household-level records must remain protected. Models used for these purposes should report error across regions and income groups. An algorithm that performs well on average can still deepen a service gap if its mistakes fall on people with fewer alternatives.

Actions that make shared agricultural data useful

Australia does not need a single vast database that attempts to hold every farm record. A practical programme can build from well-maintained public datasets, standard interfaces and clear rules for sensitive information. It should reward reuse, document failures as well as successes, and connect investment to decisions that farmers, communities and agencies can actually make.

• Publish a maintained baseline of open environmental, production, market and food-access datasets, with common identifiers, provenance and clear reuse licences.
• Establish shared vocabularies and formats for weather, water, soils, crops, livestock, pests, prices and logistics so records can be joined across systems.
• Require APIs, bulk downloads and version histories for high-value datasets, with quality notes that explain coverage, gaps, revisions and known limitations.
• Protect farm, household and Indigenous information through consent, aggregation, data minimisation and community-controlled governance for culturally sensitive holdings.
• Co-design services with grain growers, graziers, contractors, processors, food-rescue groups, remote communities and researchers, including people with limited digital access.
• Evaluate releases by measurable outcomes such as earlier pest warnings, better water allocation, lower spoilage, improved freight planning and stronger access to affordable food.

The programme can begin with shared data products that solve visible problems: a public pest alert tied to local extension advice, an open water-allocation feed for basin communities, or a food-loss dataset that helps processors and charities move surplus. Each product should include an owner, a maintenance schedule, an access standard and a public record of user feedback.

Success should be judged by use and outcomes rather than the volume of data published. A farm dashboard that changes a sowing decision, an emergency service that reaches isolated households, or a market report that helps a small processor plan inventory demonstrates value. For Australia, the test of open agricultural data is whether shared information travels quickly enough to strengthen everyday food decisions and withstand the shocks that shape production.