Reproducibility on the agenda: how IASSIST 2017 took it on

For a community that spends its days arguing about file formats, the move from spreadsheet wrangling to reproducibility strategy is a notable shift. IASSIST 2017, hosted at the University of Kansas in Lawrence from 23 to 26 May 2017, made that shift explicit, framing reproducibility not as an academic talking point but as a craft with practical, addressable problems.

The conference, organised under the banner "Data in the Middle: The Common Language of Research," gathered data librarians, archivists, domain scientists, and policy specialists from social science, agriculture, and the digital humanities. The recurring thread was that reproducibility rarely fails because researchers are careless. It fails because the surrounding infrastructure, from metadata and identifiers through documentation and archiving, is treated as somebody else's problem, or no one's at all. The Lawrence sessions tried to change that.

For researchers watching from Australia, the conversations in Kansas landed closer to home than the 14,000-kilometre flight might suggest. The same fragmented metadata and unclear licensing that complicate American replication efforts complicate work at CSIRO, the Australian Research Data Commons, and the Group of Eight universities. Anyone who has tried to chase a dataset across half a dozen university silos in Sydney, Melbourne, or Hobart will recognise the pattern.

The reproducibility squeeze on modern research

Reproducibility stopped being a niche concern around the early 2010s, when high-profile replication failures in psychology, economics, and medicine drew sustained attention. The IASSIST community treated those headlines as a professional issue rather than a moral panic. When a dataset cannot be located, the underlying paper is effectively unverifiable, regardless of how careful the original authors were.

Speakers at Lawrence framed the squeeze in concrete terms. A finding might be statistically reproducible in the narrow sense, with the same code, the same numbers, and the same output, yet still fail the broader test of being reconstructible by an independent team with reasonable effort. That distinction matters because the people doing the reconstruction are often at smaller institutions, in early-career positions, or in countries with thinner research infrastructure. Reproducibility has a global equity dimension.

The data-stewardship lens that IASSIST brings is therefore not optional. Without librarians, archivists, and repository managers actively building the scaffolding of readme files, controlled vocabularies, and persistent identifiers, the rest of the machinery grinds to a halt. The conference leaned into that view, treating reproducibility as an institutional responsibility rather than an individual virtue.

Workshops that put tools in hands

The pre-conference programme ran for two days before the main plenary sessions opened, and the workshop track carried much of the methodological weight. Sessions covered R Markdown for literate programming, Jupyter notebooks for sharing analytic pipelines, OpenRefine for cleaning messy tabular sources, and Python for reproducible scraping workflows. None of these tools are novel now, but in 2017 they were still being negotiated as the new normal in social-science data work.

What made the workshops distinctive was the audience composition. The rooms were full of data librarians, repository managers, and subject specialists from anthropology, political science, and agricultural economics rather than only statisticians. The mixing produced useful cross-talk: a librarian from a Scandinavian university might describe how her institution negotiates restricted-access contracts, while a domain scientist from a US land-grant college explains the practical realities of versioning a multi-gigabyte survey dataset.

A repeated theme was that reproducibility lives or dies at the moment of capture. Once a dataset has been stripped of its original variable labels, lost its codebook, or been saved in a proprietary format that no longer opens, no amount of downstream tooling will recover it. The workshops therefore spent considerable time on the upstream practices of file naming, directory structure, and version control that determine whether a project is reproducible a year later, let alone a decade.

Plenary voices and a common language

The plenary programme pulled in speakers who could talk across disciplinary boundaries. The framing of "data in the middle" was deliberate. Data sits between the researcher who produces it, the analyst who interprets it, and the public who may eventually scrutinise it. Treating data as a shared object rather than a private artefact is, in this view, the precondition for trustworthy research.

Presentations touched on open government data, indigenous data sovereignty, the ethics of reusing human-subjects data, and the practical politics of data citation. Several speakers returned to the point that data citation only works if datasets have stable identifiers and clear licensing. A paper that says "data available upon request" is, functionally, a paper whose findings cannot be checked by an independent reader.

The most useful plenary sessions for international attendees were those that translated high-level principles into operational checklists. What does good documentation actually look like? How granular should a readme be? When is a data dictionary overkill, and when is it essential? The IASSIST community has long been good at converting abstract standards into working templates, and that translation work was on display throughout the plenary tracks.

Digital agriculture and global food security

A surprising share of the programme was devoted to digital agriculture, reflecting the host region's strength in agronomic research and the global stakes around food systems. Sessions explored satellite-derived yield estimates, sensor networks on commercial farms, genomic data for crop breeding, and household-survey data on food security in low-income settings. Each of these generates datasets whose reproducibility carries real-world consequences.

Speakers from international agricultural research centres described the practical challenge of merging soil-moisture data from one project, fertiliser-application records from another, and remote-sensing imagery from a third, often collected under different protocols and licensing regimes. The reproducibility problem in this domain is not merely academic: a model that fails to replicate because of an undocumented transformation step might misinform planting decisions for millions of smallholders.

Sessions also discussed the FAIR principles, Findable, Accessible, Interoperable, and Reusable, as applied to agricultural data. The principles were being put through their paces in agronomic contexts, where data comes from heterogeneous sources and crosses national boundaries. The conversations offered a useful reality check on how well the principles travel from the library world into the paddock.

Metadata, standards, and the FAIR turn

The metadata thread at IASSIST 2017 ran through almost every session, but it received its own dedicated conversations. Data librarians have been pushing standardised metadata for decades, and the conference surfaced both the gains and the frustrations. Gains: schema.org, DataCite, Dublin Core, and DDI have all become widely adopted. Frustrations: adoption is uneven, and many datasets still arrive in repositories with minimal descriptive scaffolding.

A practical focus was the move toward assigning persistent identifiers to datasets, software releases, and even individual figures. Citation practices are catching up, with journals beginning to require data availability statements and the underlying identifiers. The wider point was that reproducibility cannot be an afterthought once a paper is written; it has to be designed in from the first grant proposal.

Conference materials and recordings were made openly available through the IASSIST AIR archive, allowing attendees and the wider community to revisit sessions, share slides with colleagues who could not travel, and reuse the example datasets that several presenters released. That openness itself modelled a reproducibility ethic, treating the conference's own outputs as citable, preservable objects rather than disposable event ephemera.

What Australian researchers brought back

For delegates who made the long haul from Australia, the value of IASSIST 2017 lay less in any single tool than in the collegial network it built. The Australian Research Data Commons, CSIRO, and several Group of Eight institutions, including the Australian National University in Canberra and the University of Melbourne, have long punched above their weight in research data infrastructure. NCRIS-funded projects such as the Australian National Data Service have built considerable capacity in metadata and repository work.

The conference offered a chance to test that capacity against international peers. Australian researchers tend, in this writer's experience, to be straightforward about what works and what does not. Conversations over a flat white in the Kansas conference centre sometimes turned to the peculiarities of Australian data: the multi-jurisdictional complexity of state and federal health records, the challenges of preserving indigenous data according to community-determined protocols, and the resource constraints that affect regional universities far from the capital-city research hubs.

Several Australians came home with concrete plans. A librarian from a regional university might commit to piloting a new metadata schema. A domain scientist might propose a data-management plan template that better reflects how their team actually works in the field. A repository manager might reach out to an international counterpart about cross-institutional preservation. Small steps, but reproducible scholarship is built from exactly these kinds of small, careful, documented actions.

Approach	Strengths	Weaknesses	Best fit
Open repository with DOI	Citable, persistent, widely understood	Requires curator time and metadata discipline	Published research datasets
Restricted access with request workflow	Supports sensitive or licensed data	Slow, opaque, breaks computational reproducibility	Human subjects, indigenous data, commercial data
Living dataset with versioned releases	Captures updates over time	More complex infrastructure, version confusion	Longitudinal surveys, sensor networks
Code and data on institutional Git	Familiar to technical users	Brittle without curation, identifier instability	Computational research, methods papers
Domain-specific archive such as ICPSR	Strong metadata standards, community trust	Limited to one discipline, narrow interoperability	Survey data, genomic data

Reproducibility habits worth picking up from the Lawrence gathering:

• Treat the readme as part of the scientific record, not an afterthought
• Assign a persistent identifier to every dataset before publication
• Pin software versions and document the computing environment
• Write a data-management plan that reflects what the team actually does, not an idealised version
• Cite data the way you cite papers, including creators, year, and identifier

Pointers for researchers working in Australia:

• ARDC metadata tools and training programmes are freely available at Australian member institutions
• ANDS, now folded into ARDC, maintained a rich collection of data-management templates that remain useful starting points
• State libraries and the National Library of Australia offer preservation pathways that complement institutional repositories
• Cross-institutional data-sharing agreements are easier to negotiate when the underlying metadata is already standardised
• The Australian open-access policy environment, while still uneven, offers concrete hooks for citing and preserving research data