Publications

Published Articles (38)

1. Assessing Utility of Differential Privacy for RCTs

   Kaitlyn R. Webb, Soumya Mukherjee, Aratrika Mustafi, and 2 more authors

   submitted, R&R, 2026

   Abs Scholar (6)

   Randomized controlled trials (RCTs) have become powerful tools for assessing the impact of interventions and policies in many contexts. They are considered the gold standard for causal inference in the biomedical fields and many social sciences. Researchers have published an increasing number of studies that rely on RCTs for at least part of their inference. These studies typically include the response data that has been collected, de-identified, and sometimes protected through traditional disclosure limitation methods. In this paper, we empirically assess the impact of privacy-preserving synthetic data generation methodologies on published RCT analyses by leveraging available replication packages (research compendia) in economics and policy analysis. We implement three privacy-preserving algorithms, that use as a base one of the basic differentially private (DP) algorithms, the perturbed histogram, to support the quality of statistical inference. We highlight challenges with the straight use of this algorithm and the stability-based histogram in our setting and described the adjustments needed. We provide simulation studies and demonstrate that we can replicate the analysis in a published economics article on privacy-protected data under various parameterizations. We find that relatively straightforward (at a high-level) privacy-preserving methods influenced by DP techniques allow for inference-valid protection of published data. The results have applicability to researchers wishing to share RCT data, especially in the context of low- and middle-income countries, with strong privacy protection.

2. Improving Results Reporting in RCT Registries

   Jack Cavanagh, Sarah Kopper, and Lars Vilhuber

   submitted, 2026
3. A Simulated Reconstruction and Reidentification Attack on the 2010 U.S. Census

   John M. Abowd, Tamara Adams, Robert Ashmead, and 12 more authors

   Harvard Data Science Review, Jul 2025

   DOI URL Scholar (6) Dimensions (0)
4. TROV - A Model and Vocabulary for Describing Transparent Research Objects

   Meng Li, Timothy McPhillips, Craig Willis, and 6 more authors

   International Journal of Digital Curation, Feb 2025

   Abs DOI URL Scholar (0) Dimensions (0)

   The Transparent Research Object Vocabulary (TROV) is a key element of the Transparency Certified (TRACE) approach to ensuring research trustworthiness. In contrast with methods that entail repeating computations in part or in full to verify that the descriptions of methods included in a publication are sufficient to reproduce reported results, the TRACE approach depends on a controlled computing environment termed a Transparent Research System (TRS) to guarantee that accurate, sufficiently complete, and otherwise trustworthy records are captured when results are obtained in the first place. Records identifying (1) the digital artifacts and computations that yielded a research result, (2) the TRS that witnessed the artifacts and supervised the computations, and (3) the specific conditions enforced by the TRS that warrant trust in these records, together constitute a Transparent Research Object (TRO). Digital signatures provided by the TRS and by a trusted third-party timestamp authority (TSA) guarantee the integrity and authenticity of the TRO. The controlled vocabulary TROV provides means to declare and query the properties of a TRO, to enumerate the dimensions of trustworthiness the TRS asserts for a TRO, and to verify that each such assertion is warranted by the documented capabilities of the TRS. Our approach for describing, publishing, and working with TROs imposes no restrictions on how computational artifacts are packaged or otherwise shared, and aims to be interoperable with, rather than to replace, current and future Research Object standards, archival formats, and repository layouts.

5. Using Containers to Validate Research on Confidential Data at Scale

   Lars Vilhuber

   Harvard Data Science Review, Jun 2025

   Abs DOI URL Scholar (1) Dimensions (0)

   I describe past experience with the validation server process over 10 years and several hundred users, as a means to provide proxy access to confidential data. As a modern replacement, I propose the use of containers—simulated computers that encapsulate entire software and file structures. The use of containers ensures reproducibility, reliable portability, and enables scalability. Infrastructure can be outsourced to commercial providers or users, at little to no cost to data providers. The only likely limitation to full automation is the absence of automated output vetting algorithms at statistical agencies.

6. Reproduce to validate: A comprehensive study on the reproducibility of economics research

   Sylvérie Herbert, Hautahi Kingi, Flavio Stanchi, and 1 more author

   Canadian Journal of Economics/Revue canadienne d’économique, Aug 2024

   Abs DOI URL OA? Scholar (5) Dimensions (3)

   Abstract Journals have pushed for transparency of research through data availability policies. Such data policies improve availability of data and code, but what is the impact on reproducibility? We present results from a large reproduction exercise for articles published in the American Economic Journal: Applied Economics, which has had a data availability policy since its inception in 2009. Out of 363 published articles, we assessed 274 articles. All articles provided some materials. We excluded 122 articles that required confidential or proprietary data or that required the replicator to otherwise obtain the data (44.5% of assessed articles). We attempted to reproduce 152 articles and were able to fully reproduce the results of 68 (44.7% of attempted reproductions). A further 66 (43.4% of attempted reproductions) were partially reproduced. Many articles required complex code changes even when at least partially reproduced. We collect bibliometric characteristics of authors, but find no evidence for author characteristics as determinants of reproducibility. There does not appear to be a citation bonus for reproducibility. The data availability policy of this journal was effective to ensure availability of materials, but is insufficient to ensure reproduction without additional work by replicators. , Résumé Les journaux militent pour la transparence de la recherche par le biais de politiques sur la disponibilité des données . De telles politiques sur les données améliorent la disponibilité des données et du code, mais quelle en est l’incidence sur la reproductibilité? Nous présentons les résultats d’un grand exercice de reproduction pour des articles publiés dans le American Economic Journal: Applied Economics , qui a une politique de disponibilité des données depuis sa création en 2009. Parmi les 363 articles publiés, nous en évaluons 274. Tous les articles avaient fourni certains documents. Nous avons exclu 122 articles qui nécessitaient des données confidentielles ou exclusives, ou qui exigeaient que la personne chargée de la réplication obtienne autrement les données (44,5% des articles évalués). Nous avons tenté de reproduire 152 articles et avons réussi à entièrement reproduire les résultats de 68 articles (44,7% des reproductions tentées). Nous avons aussi partiellement reproduit 66 autres articles (43,4% des reproductions tentées). De nombreux articles nécessitaient de complexes changements de code, même lorsqu’ils étaient au moins partiellement reproduits. Nous avons recueilli les caractéristiques bibliométriques des auteurs, mais n’avons pas constaté qu’elles étaient déterminantes pour la reproductibilité. La reproductibilité ne semble offrir aucun avantage en matière de citations. La politique de disponibilité des données de ce journal a été efficace pour assurer la disponibilité des documents, mais insuffisante pour assurer la reproduction sans travail supplémentaire des personnes chargées de la réplication.

7. A Guide for Social Science Journal Editors on Easing into Open Science

   Priya Silverstein, Colin Elman, Amanda Montoya, and 36 more authors

   Research Integrity and Peer Review, Feb 2024

   Abs DOI Scholar (0) Dimensions (23)

   Journal editors have a large amount of power to advance open science in their respective fields by incentivizing and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org) has collated several resources on embedding open science in journal editing (www.dpjedi.org/resources). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we have created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx)).

8. Introduction to the special issue: Models of linked employer–employee data: Twenty years after “High wage workers and high wage firms”

   David Card, Ian Schmutte, and Lars Vilhuber

   Journal of Econometrics, 2023

   DOI URL OA? Dimensions (1)
9. Reproducibility and transparency versus privacy and confidentiality: Reflections from a data editor

   Lars Vilhuber

   Journal of Econometrics, Jun 2023

   Published online

   DOI URL OA? Scholar (18) Dimensions (6)
10. Reinforcing Reproducibility and Replicability: An Introduction

    Lars Vilhuber, Ian Schmutte, Aleksandr Michuda, and 1 more author

    Harvard Data Science Review, Jul 2023

    DOI URL Scholar (0) Dimensions (1)
11. An Interview with John M. Abowd

    Ian Schmutte and Lars Vilhuber

    International Statistical Review, Feb 2022

    DOI URL OA? Scholar (0) Dimensions (1)
12. Teaching for large-scale Reproducibility Verification

    Lars Vilhuber, Hyuk Harry Son, Meredith Welch, and 2 more authors

    Journal of Statistics and Data Science Education, Sep 2022

    Abs DOI URL Scholar (18) Dimensions (15)

    We describe a unique environment in which undergraduate students from various STEM and social science disciplines are trained in data provenance and reproducible methods, and then apply that knowledge to real, conditionally accepted manuscripts and associated replication packages. We describe in detail the recruitment, training, and regular activities. While the activity is not part of a regular curriculum, the skills and knowledge taught through explicit training of reproducible methods and principles, and reinforced through repeated application in a real-life workflow, contribute to the education of these undergraduate students, and prepare them for post-graduation jobs and further studies.

13. On privacy in the age of COVID-19

    Cynthia Dwork, Alan Karr, Kobbi Nissim, and 1 more author

    Journal of Privacy and Confidentiality, Feb 2021

    DOI URL Scholar (0) Dimensions (7)
14. Recalculating ... : How Uncertainty in Local Labour Market Definitions Affects Empirical Findings

    Andrew Foote, Mark J. Kutzbach, and Lars Vilhuber

    Applied Economics, Jan 2021

    DOI URL OA? Scholar (16) Dimensions (12)
15. metajelo: A metadata package for journals to support external linked objects

    Carl Lagoze and Lars Vilhuber

    International Journal of Digital Curation, 2021

    DOI Scholar (0) Dimensions (0)
16. Applying data synthesis for longitudinal business data across three countries

    M. Jahangir Alam, Benoit Dostie, Jörg Drechsler, and 1 more author

    Statistics in Transition New Series, 2020

    DOI URL OA? Scholar (6) Dimensions (5)
17. Total Error and Variability Measures for the Quarterly Workforce Indicators and Lehd Origin-Destination Employment Statistics in Onthemap

    Kevin L McKinney, Andrew S Green, Lars Vilhuber, and 1 more author

    Journal of Survey Statistics and Methodology, Nov 2020

    Abs DOI URL OA? Scholar (13) Dimensions (10)

    Abstract We report results from the first comprehensive total quality evaluation of five major indicators in the U.S. Census Bureau’s Longitudinal Employer-Household Dynamics (LEHD) Program Quarterly Workforce Indicators (QWI): total flow-employment, beginning-of-quarter employment, full-quarter employment, average monthly earnings of full-quarter employees, and total quarterly payroll. Beginning-of-quarter employment is also the main tabulation variable in the LEHD Origin-Destination Employment Statistics (LODES) workplace reports as displayed in OnTheMap (OTM), including OTM for Emergency Management. We account for errors due to coverage; record-level non-response; edit and imputation of item missing data; and statistical disclosure limitation. The analysis reveals that the five publication variables under study are estimated very accurately for tabulations involving at least 10 jobs. Tabulations involving three to nine jobs are a transition zone, where cells may be fit for use with caution. Tabulations involving one or two jobs, which are generally suppressed on fitness-for-use criteria in the QWI and synthesized in LODES, have substantial total variability but can still be used to estimate statistics for untabulated aggregates as long as the job count in the aggregate is more than 10.

18. Reproducibility and Replicability in Economics

    Lars Vilhuber

    Harvard Data Science Review, Dec 2020

    DOI URL Scholar (65) Dimensions (30)
19. Effects of a Government-Academic Partnership: Has the NSF-Census Bureau Research Network Helped Improve the US Statistical System?

    Daniel H. Weinberg, John M. Abowd, Robert F. Belli, and 13 more authors

    Journal of Survey Statistics and Methodology, 2019

    First published December 2018

    Abs DOI URL OA? Scholar (8) Dimensions (3)

    Abstract. The National Science Foundation-Census Bureau Research Network (NCRN) was established in 2011 to create interdisciplinary research nodes on methodolo

20. Remembering Stephen Fienberg

    Aleksandra Slavković and Lars Vilhuber

    Journal of Privacy and Confidentiality, Dec 2018

    DOI Scholar (2) Dimensions (1)
21. Relaunching the Journal of Privacy and Confidentiality

    Lars Vilhuber

    Journal of Privacy and Confidentiality, Dec 2018

    DOI Scholar (4) Dimensions (0)
22. Understanding the effect of procedural justice on psychological distress

    Julie Cloutier, Lars Vilhuber, Denis Harrisson, and 1 more author

    International Journal of Stress Management, 2017

    Abs DOI OA? Dimensions (18)

    Studies on the effect of procedural justice on psychological distress present conflicting results. Drawing on instrumental and relational perspectives of justice, we test the hypothesis that the perception of procedural justice influences the level of workers’ psychological distress. Using a number of validated instruments to collected data from 659 workers in three call centers, we use OLS regressions and Hayes’ PROCESS tool to show that the perception of procedural justice has a direct, unique, and independent effect on psychological distress. The perception of procedural justice has no instrumental role, the key mechanism being the relational role, suggesting that perceived injustice influences psychological distress because it threatens self-esteem. Distributive justice perceptions (recognition, promotions, job security) are not associated with psychological distress, calling into question Siegrist’s model. Our findings suggest that perceived procedural justice provides workers better evidence of the extent to which they are valued and appreciated members of their organizations than do perceptions of distributive justice. The results highlight the greater need for workers to be valued and appreciated for who they are (consideration and esteem), rather than for what they do for their organization (distributive justice of rewards).

23. Making confidential data part of reproducible research

    C Lagoze and L Vilhuber

    Chance, 2017

    Abs URL Scholar (1)

    The rise of data-centric research practices has uncovered shortcomings in the traditional scholarly communication system. The foundation of that system, the peer-reviewed publication,“[the] selective distribution of ink on paper, or… electronic facsimiles of the same”(Bourne, et al., 2011), does not adequately support what has become an essential element of scholarship; the reproducibility of research results. That is, duplicating a ...

24. Using partially synthetic microdata to protect sensitive cells in business statistics

    Javier Miranda and Lars Vilhuber

    Statistical Journal of the International Association for Official Statistics, 2016

    Abs DOI URL OA? Scholar (10) Dimensions (4)

    We describe and analyze a method that blends records from both observed and synthetic microdata into public-use tabulations on establishment statistics. The resulting tables use synthetic data only in potentially sensitive cells. We describe different algorithms, and present preliminary results when applied to the Census Bureau’s Business Dynamics Statistics and Synthetic Longitudinal Business Database, highlighting accuracy and protection afforded by the method when compared to existing public-use tabulations (with suppressions).

25. Synthetic establishment microdata around the world

    Lars Vilhuber, John M. Abowd, and Jerome P. Reiter

    Statistical Journal of the International Association for Official Statistics, 2016

    Abs DOI OA? Scholar (19) Dimensions (11)

    In contrast to the many public-use microdata samples available for individual and household data from many statistical agencies around the world, there are virtually no establishment or firm microdata available. In large part, this difficulty in providing access to business micro data is due to the skewed and sparse distributions that characterize business data. Synthetic data are simulated data generated from statistical models. We organized sessions at the 2015 World Statistical Congress and the 2015 Joint Statistical Meetings, highlighting work on synthetic establishment microdata. This overview situates those papers, published in this issue, within the broader literature.

26. Looking back on three years of using the Synthetic LBD beta

    Javier Miranda and Lars Vilhuber

    Statistical Journal of the IAOS: Journal of the International Association for Official Statistics, 2014

    Abs DOI URL OA? Dimensions (0)

    Distributions of business data are typically much more skewed than those for household or individual data and public knowledge of the underlying units is greater. As a results, national statistical offices (NSOs) rarely release establishment or firm-level business microdata due to the risk to respondent confidentiality. One potential approach for overcoming these risks is to release synthetic data where the establishment data are simulated from statistical models designed to mimic the distributions of the real underlying microdata. The US Census Bureau?s Center for Economic Studies in collaboration with Duke University, the National Institute of Statistical Sciences, and Cornell University made available a synthetic public use file for the Longitudinal Business Database (LBD) comprising more than 20 million records for all business establishment with paid employees dating back to 1976. The resulting product, dubbed the SynLBD, was released in 2010 and is the first-ever comprehensive business microdata set publicly released in the United States including data on establishments employment and payroll, birth and death years, and industrial classification. This pa- per documents the scope of projects that have requested and used the SynLBD.

27. A First Step Towards A German SynLBD: Constructing A German Longitudinal Business Database

    Jörg Drechsler and Lars Vilhuber

    Statistical Journal of the IAOS: Journal of the International Association for Official Statistics, 2014

    Abs DOI URL OA? Dimensions (2)

    One major criticism against the use of synthetic data has been that the efforts necessary to generate useful synthetic data are so in- tense that many statistical agencies cannot afford them. We argue many lessons in this evolving field have been learned in the early years of synthetic data generation, and can be used in the development of new synthetic data products, considerably reducing the required in- vestments. The final goal of the project described in this paper will be to evaluate whether synthetic data algorithms developed in the U.S. to generate a synthetic version of the Longitudinal Business Database (LBD) can easily be transferred to generate a similar data product for other countries. We construct a German data product with infor- mation comparable to the LBD - the German Longitudinal Business Database (GLBD) - that is generated from different administrative sources at the Institute for Employment Research, Germany. In a fu- ture step, the algorithms developed for the synthesis of the LBD will be applied to the GLBD. Extensive evaluations will illustrate whether the algorithms provide useful synthetic data without further adjustment. The ultimate goal of the project is to provide access to multiple synthetic datasets similar to the SynLBD at Cornell to enable comparative studies between countries. The Synthetic GLBD is a first step towards that goal.

28. Differential privacy applications to bayesian and linear mixed model estimation

    John M. Abowd, Matthew J. Schneider, and Lars Vilhuber

    Journal of Privacy and Confidentiality, 2013

    Abs DOI URL Dimensions (0)

    We consider a particular maximum likelihood estimator (MLE) and a computationally intensive Bayesian method for differentially private estimation of the linear mixed-effects model (LMM) with normal random errors. The LMM is important because it is used in small-area estimation and detailed industry tabulations that present significant challenges for confidentiality protection of the underlying data. The differentially private MLE performs well compared to the regular MLE, and deteriorates as the protection increases for a problem in which the small-area variation is at the county level. More dimensions of random effects are needed to adequately represent the time dimension of the data, and for these cases the differentially private MLE cannot be computed. The direct Bayesian approach for the same model uses an informative, reasonably diffuse prior to compute the posterior predictive distribution for the random effects. The empirical differential privacy of this approach is estimated by direct computation of the relevant odds ratios after deleting influential observations according to various criteria.

29. Data Management of Confidential Data

    Carl Lagoze, William C. Block, Jeremy Williams, and 2 more authors

    International Journal of Digital Curation, 2013

    DOI Scholar (22) Dimensions (9)
30. Did the Housing Price Bubble Clobber Local Labor Market Job and Worker Flows When It Burst?

    John M Abowd and Lars Vilhuber

    The American Economic Review, May 2012

    Abs DOI URL OA? Scholar (17) Dimensions (7)

    We use the Census Bureau’s Quarterly Workforce Indicators and the Federal Housing Finance Agency’s House Price Indices to study the effects of the housing price bubble on local labor markets. We show that the 35 MSAs in the top decile of the house price boom were most severely impacted. Their stable job employment fell much more than the national average. Their real wage rates did not fall as fast as the national average. Accessions fell much faster than average while separations were constant. Job creations fell substantially while destructions rose slightly.

31. National estimates of gross employment and job flows from the quarterly workforce indicators with demographic and industry detail

    John M. Abowd and Lars Vilhuber

    Journal of Econometrics, 2011

    Free to read at https://pmc.ncbi.nlm.nih.gov/articles/PMC3079891/

    Abs DOI Scholar (70) Dimensions (28)

    The Quarterly Workforce Indicators (QWI) are local labor market data produced and released every quarter by the United States Census Bureau. Unlike any other local labor market series produced in the US or the rest of the world, QWI measure employment flows for workers (accession and separations), jobs (creations and destructions) and earnings for demographic subgroups (age and gender), economic industry (NAICS industry groups), detailed geography (block (experimental), county, Core-Based Statistical Area, and Workforce Investment Area), and ownership (private, all) with fully interacted publication tables. The current QWI data cover 47 states, about 98% of the private workforce in those states, and about 92% of all private employment in the entire economy. State participation is sufficiently extensive to permit us to present the first national estimates constructed from these data. We focus on worker, job, and excess (churning) reallocation rates, rather than on levels of the basic variables. This permits a comparison to existing series from the Job Openings and Labor Turnover Survey and the Business Employment Dynamics Series from the Bureau of Labor Statistics (BLS). The national estimates from the QWI are an important enhancement to existing series because they include demographic and industry detail for both worker and job flow data compiled from underlying micro-data that have been integrated at the job and establishment levels by the Longitudinal Employer-Household Dynamics Program at the Census Bureau. The estimates presented herein were compiled exclusively from public-use data series and are available for download.

32. Science, confidentiality, and the public interest

    John M. Abowd and Lars Vilhuber

    Chance, 2011

    DOI URL OA? Scholar (9) Dimensions (0)
33. Procedural justice criteria in salary determination

    Julie Cloutier and Lars Vilhuber

    Journal of Managerial Psychology, 2008

    tex.eprint: https://doi.org/10.1108/02683940810894765

    Abs DOI URL OA? Scholar (74) Dimensions (25)

    Purpose – The purpose of this research is to identify the dimensionality of the procedural justice construct and the criteria used by employees to assess procedural justice, in the context of salary determination.Design/methodology/approach – Based on a survey of 297 Canadian workers, the paper uses confirmatory factor analysis (CFA) to test the dimensionality and the discriminant and convergent validity of our procedural justice construct. Convergent and predictive validity are also tested using hierarchical linear regressions.Findings – The paper shows the multidimensionality of the procedural justice construct: justice of the salary determination process is assessed through the perceived characteristics of allocation procedures, the perceived characteristics of decision‐makers, and system transparency.Research limitations/implications – Results could be biased towards acceptance; this is discussed. The results also suggest possible extensions to the study.Practical implications – Knowledge of the justice standards improves the ability of organizations to effectively manage the salary determination process and promote its acceptance among employees. Emphasizes the need to adequately manage the selection, training, and perception of decision makers.Originality/value – The paper identifies the standards of procedural justice for salary determination processes. It contributes to the theoretical literature by providing a new multidimensional conceptualization, which helps to better understand the psychological process underlying the perception of procedural justice. The presence of a dimension associated with decision makers is novel and critical for compensation studies.

34. The sensitivity of economic statistics to coding errors in personal identifiers

    John M. Abowd and Lars Vilhuber

    Journal of Business & Economic Statistics, Apr 2005

    URL Scholar (87)
35. Escaping poverty for low-wage workers: The role of employer characteristics and changes

    Harry Holzer, Julia Lane, and Lars Vilhuber

    Industrial and Labor Relations Review, Jul 2004

    tex.alturl: http://www.jstor.org/stable/4126683 tex.owner: vilhuber

    Abs DOI URL OA? Scholar (5) Dimensions (16)

    Using a unique dataset based on individual Unemployment Insurance wage records for Illinois in the 1990s that are matched to other Census data, the authors analyze the extent to which escape from or entry into low earnings among adult workers was associated with changes in their employers and firm characteristics. The results show considerable mobility into and out of low earnings status, even for adults. They indicate that job changes were an important part of the process by which workers escaped or entered low-wage status, and that changes in employer characteristics help to account for these job changes. Matches between personal and firm characteristics also contributed to observed earnings outcomes.

36. Early career experiences and later career outcomes: Comparing the United States, France and Germany

    David N. Margolis, Véronique Simonnet, and Lars Vilhuber

    Vierteljahrshefte zur Wirtschaftsforschung, 2001

    Abs DOI URL Dimensions (5)

    This paper explores the links between individuals’ early career experiences and their labor market outcomes 5 to 20 years later using data from France, (western) Germany, and the United States. Relative to most of the literature, we consider a large set of measures of men’s early career experiences and later career outcomes. Our results differ significantly across countries. Labor market outcomes in Germany are consistent with a dual labor market model. In the case of American workers, either the market learns about unobservable worker characteristics over time or the implicit contracts established at the start of the career are increasingly renegotiated over time. Unobserved heterogeneity in individuals’ networks of labor market contacts is consistent with our results for France. These results reflect optimal firm responses to the different institutional environments in each country in the presence of ex ante imperfect information concerning young workers.

37. La spécificité de la formation en milieu de travail : un survol des contributions théoriques et empiriques récentes,

    Lars Vilhuber

    L’Actualité économique, Revue d’analyse économique, Mar 2001
38. Continuous Training and sectoral mobility in Germany: Evidence from the 90s

    Lars Vilhuber

    Vierteljahresheft für Wirtschaftsforschung, 1999

    Abs URL

    see Vilhuber99b

Working Papers (68)

1. Assessing Utility of Differential Privacy for RCTs

   Kaitlyn R. Webb, Soumya Mukherjee, Aratrika Mustafi, and 2 more authors

   arXiv arxiv:2309.14581v2, 2026

   Version Number: 2

   Abs DOI URL Scholar (6) Dimensions (0)

   Randomized controlled trials (RCTs) have become powerful tools for assessing the impact of interventions and policies in many contexts. They are considered the gold standard for causal inference in the biomedical fields and many social sciences. Researchers have published an increasing number of studies that rely on RCTs for at least part of their inference. These studies typically include the response data that has been collected, de-identified, and sometimes protected through traditional disclosure limitation methods. In this paper, we empirically assess the impact of privacy-preserving synthetic data generation methodologies on published RCT analyses by leveraging available replication packages (research compendia) in economics and policy analysis. We implement three privacy-preserving algorithms, that use as a base one of the basic differentially private (DP) algorithms, the perturbed histogram, to support the quality of statistical inference. We highlight challenges with the straight use of this algorithm and the stability-based histogram in our setting and described the adjustments needed. We provide simulation studies and demonstrate that we can replicate the analysis in a published economics article on privacy-protected data under various parameterizations. We find that relatively straightforward (at a high-level) privacy-preserving methods influenced by DP techniques allow for inference-valid protection of published data. The results have applicability to researchers wishing to share RCT data, especially in the context of low- and middle-income countries, with strong privacy protection.

2. Assessing Reproducibility in Economics Using Standardized Crowd-sourced Analysis

   Abel Brodeur, Seung Yong Sung, Edward Miguel, and 2 more authors

   National Bureau of Economic Research, Working Paper 33753, May 2025

   Abs DOI URL OA? Scholar (1) Dimensions (0)

   This paper presents a framework to standardize crowd-sourced computational reproductions in economics through the Social Science Reproduction Platform (SSRP). The approach address four main challenges for computational reproductions: a lack of standardization, aggregation issues, existing incentives for “adversarial” interactions, and the loss of knowledge from analyses that are never published. We then summarize the first 487 reproductions uploaded on the SSRP. The results show substantial heterogeneity in the ability to successfully reproduce empirical results in economics research, with approximately 30% of recent studies meeting at least a basic definition of being computationally reproducible.

3. Mass Reproducibility and Replicability: A New Hope

   Abel Brodeur, Derek Mikola, Nikolai Cook, and 346 more authors

   I4R Discussion Paper Series, Working Paper 107, 2024

   Abs URL Scholar (65)

   This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness assessments. It reveals several patterns. First, we uncover a high rate of fully computationally reproducible results (over 85%). Second, excluding minor issues like missing packages or broken pathways, we uncover coding errors for about 25% of studies, with some studies containing multiple errors. Third, we test the robustness of the results to 5,511 re-analyses. We find a robustness reproducibility of about 70%. Robustness reproducibility rates are relatively higher for re-analyses that introduce new data and lower for re-analyses that change the sample or the definition of the dependent variable. Fourth, 52% of re-analysis effect size estimates are smaller than the original published estimates and the average statistical significance of a re-analysis is 77% of the original. Lastly, we rely on six teams of researchers working independently to answer eight additional research questions on the determinants of robustness reproducibility. Most teams find a negative relationship between replicators’ experience and reproducibility, while finding no relationship between reproducibility and the provision of intermediate or even raw data combined with the necessary cleaning codes.

4. Crowdsourcing Digital Public Goods: A Field Experiment on Metadata Contributions

   Linfeng Li, Yan Chen, Margaret C. Levenstein, and 1 more author

   Social Science Research Network, SSRN Scholarly Paper 5008203, Nov 2024

   Abs DOI URL Scholar (2) Dimensions (0)

   This study explores why people choose to contribute metadata, which is data about data. Using a field experiment conducted with more than 3,000 authors of AEA journal articles, our control message reduces the uncertainty about the future value of metadata, whereas those from the treatment conditions additionally make the private or social benefits of metadata salient. Surprisingly, we find that participants in the control condition provide significantly more metadata compared to those in the treatments. This suggests that simply knowing that metadata will have value is sufficient to motivate people to contribute. Our results also highlight the importance of interface design in online field experiments.

5. Protecting Confidential Data through Non-Statistical Methods

   Lars Vilhuber

   Cornell University, Document 116054, Oct 2024

   Abs URL Scholar (2)

   This chapter will rely on and update previous overviews of how researchers, citizens, and administrators can reliably and securely access confidential data, i.e., data that cannot be simply published as “open data”. I will discuss various legal, technical, and practical ways of securing access to data that is needed for computations. This obviously depends on the type and complexity of the computations but also depends on the who, how, and where access is needed.

6. The 2010 Census Confidentiality Protections Failed, Here’s How and Why

   John M. Abowd, Tamara Adams, Robert Ashmead, and 11 more authors

   arXiv arXiv:2312.11283, Dec 2023

   arXiv:2312.11283 null

   Abs URL Scholar (32)

   Using only 34 published tables, we reconstruct five variables (census block, sex, age, race, and ethnicity) in the confidential 2010 Census person records. Using the 38-bin age variable tabulated at the census block level, at most 20.1% of reconstructed records can differ from their confidential source on even a single value for these five variables. Using only published data, an attacker can verify that all records in 70% of all census blocks (97 million people) are perfectly reconstructed. The tabular publications in Summary File 1 thus have prohibited disclosure risk similar to the unreleased confidential microdata. Reidentification studies confirm that an attacker can, within blocks with perfect reconstruction accuracy, correctly infer the actual census response on race and ethnicity for 3.4 million vulnerable population uniques (persons with nonmodal characteristics) with 95% accuracy, the same precision as the confidential data achieve and far greater than statistical baselines. The flaw in the 2010 Census framework was the assumption that aggregation prevented accurate microdata reconstruction, justifying weaker disclosure limitation methods than were applied to 2010 Census public microdata. The framework used for 2020 Census publications defends against attacks that are based on reconstruction, as we also demonstrate here. Finally, we show that alternatives to the 2020 Census Disclosure Avoidance System with similar accuracy (enhanced swapping) also fail to protect confidentiality, and those that partially defend against reconstruction attacks (incomplete suppression implementations) destroy the primary statutory use case: data for redistricting all legislatures in the country in compliance with the 1965 Voting Rights Act.

7. Assessing Utility of Differential Privacy for RCTs

   Soumya Mukherjee, Aratrika Mustafi, Aleksandra Slavković, and 1 more author

   arXiv arXiv:2309.14581v1, Sep 2023

   arXiv:2309.14581 [cs, econ, stat]

   Abs DOI URL Scholar (6) Dimensions (0)

   Randomized control trials, RCTs, have become a powerful tool for assessing the impact of interventions and policies in many contexts. They are considered the gold-standard for inference in the biomedical fields and in many social sciences. Researchers have published an increasing number of studies that rely on RCTs for at least part of the inference, and these studies typically include the response data collected, de-identified and sometimes protected through traditional disclosure limitation methods. In this paper, we empirically assess the impact of strong privacy-preservation methodology (with }ac{DP} guarantees), on published analyses from RCTs, leveraging the availability of replication packages (research compendia) in economics and policy analysis. We provide simulations studies and demonstrate how we can replicate the analysis in a published economics article on privacy-protected data under various parametrizations. We find that relatively straightforward DP-based methods allow for inference-valid protection of the published data, though computational issues may limit more complex analyses from using these methods. The results have applicability to researchers wishing to share RCT data, especially in the context of low- and middle-income countries, with strong privacy protection.

8. Reproducibility and Transparency versus Privacy and Confidentiality: Reflections from a Data Editor

   Lars Vilhuber

   arXiv, submitted version 2305.14478, 2023

   Version Number: 1

   Abs DOI URL Scholar (18) Dimensions (0)

   Transparency and reproducibility are often seen in opposition to privacy and confidentiality. Data that need to be kept confidential are seen as an impediment to reproducibility, and privacy would seem to inhibit transparency. I bring a more nuanced view to the discussion, and show, using examples from over 1,000 reproducibility assessments, that confidential data can very well be used in reproducible and transparent research. The key insight is that access to most confidential data, while tedious, is open to hundreds if not thousands of researchers. In cases where few researchers can consider accessing such data in the future, reproducibility services, such as those provided by some journals, can provide some evidence for effective reproducibility even when the same data may not be available for future research.

9. Data and Code Availability Standard

   Miklós Koren, Marie Connolly, Joan Lull, and 1 more author

   Social Science Data Editors, Dec 2022

   Version Number: 1.0

   Abs DOI URL Dimensions (0)

   DCAS is a standard for sharing research code and data, endorsed by leading journals in social sciences. It is maintained by the Social Science Data Editors.

10. An Interview with John M. Abowd

    Ian Schmutte and Lars Vilhuber

    Cornell University Labor Dynamics Institute Document, Feb 2022

    Abs URL Scholar (0)

    John M. Abowd is the Chief Scientist and Associate Director for Research and Methodology, U.S. Census Bureau. He completed his A.B. in Economics at NotreDame in 1973 and his Ph.D. in Economics at University of Chicago in 1977 under Arnold Zellner. During his academic career, John has held faculty positions at Princeton, the University of Chicago, and, since 1987 at Cornell University where he is the Edmund Ezra Day Professor Emeritus of Economics, Statistics and Data Science. John was trained as a statistician and labor economist, and his economic research has focused on the rigorous empirical evaluation of labor market institutions. In the late 1990s, he began working with the Census Bureau on projects that would end up leveraging administrative and survey records into official statistical products. Through that work, he has developed a research agenda focused on issues necessary to generate those products, including data privacy, synthetic data, total error analysis, data linkage, missing data problems, among others.

11. A template README for social science replication packages

    Lars Vilhuber, Marie Connolly, Miklós Koren, and 2 more authors

    Zenodo v1.1.0, Nov 2022

    Abs URL Scholar (9)

    The typical README in social science journals serves the purpose of guiding a reader through the available material and a route to replicating the results in the research paper, including the description of the origins of data and/or description of programs. As such, a good README file should first provide a brief overview of the available material and a brief guide as to how to proceed from beginning to end, before then diving into the specifics. These template files structure such a README in a way that is compliant with the typical data and code workflow in the social sciences.

12. Teaching for large-scale Reproducibility Verification

    Lars Vilhuber, Hyuk Harry Son, Meredith Welch, and 2 more authors

    arXiv arxiv:2204.01540v1, Mar 2022

    Abs URL Scholar (18)

    We describe a unique environment in which undergraduate students from various STEM and social science disciplines are trained in data provenance and reproducible methods, and then apply that knowledge to real, conditionally accepted manuscripts and associated replication packages. We describe in detail the recruitment, training, and regular activities. While the activity is not part of a regular curriculum, the skills and knowledge taught through explicit training of reproducible methods and principles, and reinforced through repeated application in a real-life workflow, contribute to the education of these undergraduate students, and prepare them for post-graduation jobs and further studies.

13. Applying Data Synthesis for Longitudinal Business Data across Three Countries

    M. Jahangir Alam, Benoit Dostie, Jörg Drechsler, and 1 more author

    arXiv arxiv:2008.02246, Jul 2020

    Abs DOI URL Scholar (6) Dimensions (0)

    Data on businesses collected by statistical agencies are challenging to protect. Many businesses have unique characteristics, and distributions of employment, sales, and profits are highly skewed. Attackers wishing to conduct identification attacks often have access to much more information than for any individual. As a consequence, most disclosure avoidance mechanisms fail to strike an acceptable balance between usefulness and confidentiality protection. Detailed aggregate statistics by geography or detailed industry classes are rare, public-use microdata on businesses are virtually inexistant, and access to confidential microdata can be burdensome. Synthetic microdata have been proposed as a secure mechanism to publish microdata, as part of a broader discussion of how to provide broader access to such data sets to researchers. In this article, we document an experiment to create analytically valid synthetic data, using the exact same model and methods previously employed for the United States, for data from two different countries: Canada (LEAP) and Germany (BHP). We assess utility and protection, and provide an assessment of the feasibility of extending such an approach in a cost-effective way to other data.

14. Consumer expectations around COVID-19: Evolution over time

    Fabian Lange and Lars Vilhuber

    Labor Dynamics Institute, Online, 2020

    URL
15. A template README for social science replication packages

    Lars Vilhuber, Marie Connolly, Miklós Koren, and 2 more authors

    Zenodo, Dec 2020

    Version Number: v1.0.0

    Abs DOI URL Scholar (9) Dimensions (0)

    The typical README in social science journals serves the purpose of guiding a reader through the available material and a route to replicating the results in the research paper, including the description of the origins of data and/or description of programs. As such, a good README file should first provide a brief overview of the available material and a brief guide as to how to proceed from beginning to end, before then diving into the specifics. These template files structure such a README in a way that is compliant with the typical data and code workflow in the social sciences.

16. Migrating historical AEA supplements

    Lars Vilhuber

    Cornell University, release v20200515, May 2020

    URL
17. Criminal Record Inaccuracies and the Impact of a Record Education Intervention on Employment-Related Outcomes

    Martin Wells, Erin York Cornwell, Linda Barrington, and 3 more authors

    Cornell University, Final Report, Jan 2020

    Abs URL

    More than 70 million Americans have some form of criminal record, which can limit their access to employment opportunities, eligibility for occupational licensure, and public benefits. The use of criminal background checks in the hiring process has also dramatically increased over the past decade, and there is reason to think that many criminal records are inaccurate. Prior research has not determined the extent of errors on criminal records. We also do not know educating individuals about their records may promote efforts toward record correction and improve employment and other economic outcomes. The present study harnesses a unique opportunity to investigate the accuracy of criminal records and the impact of a record education intervention on job-seeking behaviors, employment opportunities, and economic outcomes for people with criminal records. We focus on class members of the Gonzalez, et al. v. Pritzker class action lawsuit. This group of individuals applied for a job with the 2010 Census, but they were denied employment because of a criminal background check. As part of the lawsuit settlement, class members were offered the choice of one of two remedies: a criminal records intervention that educates them about their criminal record and their related employment rights, or early notice of hiring for the 2020 Census. Individuals who chose the record education intervention are provided with a copy of their criminal record and a training session to review their record and provide information about their rights when applying for jobs or other employment-related opportunities. In addition, all class members in the two remedy groups were invited to participate in the first two waves of the Cornell Criminal Records Panel Survey (CCRPS). We combine data from the panel survey with administrative data from the records training (including actual criminal records) to address two main research questions. First, we ask: What is the prevalence of errors in criminal records of members of this class, and how are these errors distributed across racial/ethnic and sociodemographic groups? Using data from the record education intervention, we describe the errors discovered on participants’ records and how those errors vary across racial/ethnic and socioeconomic groups. Second, we ask: How does understanding one’s criminal record and relevant legal rights affect job- seeking behaviors, employment opportunities, economic attainment, and social engagement? To address this question, we leverage a quasi-experimental design, comparing class members who receive the criminal records intervention to those who opt into early notice of Census 2020 hiring, in order to examine how the criminal records intervention shapes job- seeking and other behaviors.

18. Introductory Readings in Formal Privacy for Economists

    John Abowd, Ian Schmutte, William Sexton, and 1 more author

    Apr 2019

    DOI URL Dimensions (0)
19. Why the Economics Profession Must Actively Participate in the Privacy Protection Debate

    John M. Abowd, Ian M. Schmutte, William N. Sexton, and 1 more author

    Labor Dynamics Institute, Cornell University, Document 51, Jan 2019

    Abs URL Scholar (14)

    When Google or the U.S. Census Bureau publish detailed statistics on browsing habits or neighborhood characteristics, some privacy is lost for everybody while supplying public information. To date, economists have not focused on the privacy loss inherent in data publication. In their stead, these issues have been advanced almost exclusively by computer scientists who are primarily interested in technical problems associated with protecting privacy. Economists should join the discussion, first, to determine where to balance privacy protection against data quality; a social choice problem. Furthermore, economists must ensure new privacy models preserve the validity of public data for economic research.

20. Suboptimal Provision of Privacy and Statistical Accuracy When They are Public Goods

    John M. Abowd, Ian M. Schmutte, William Sexton, and 1 more author

    arXiv arxiv:1906.09353, Jun 2019

    Abs DOI URL Scholar (0) Dimensions (0)

    With vast databases at their disposal, private tech companies can compete with public statistical agencies to provide population statistics. However, private companies face different incentives to provide high-quality statistics and to protect the privacy of the people whose data are used. When both privacy protection and statistical accuracy are public goods, private providers tend to produce at least one suboptimally, but it is not clear which. We model a firm that publishes statistics under a guarantee of differential privacy. We prove that provision by the private firm results in inefficiently low data quality in this framework.

21. Cornell Criminal Records Panel Study Questionnaire Wave 2

    Linda Barrington, Esta R. Bigler, Hassan Enayati, and 3 more authors

    Cornell Criminal Records Panel Study (CRPS), Document 69333, Mar 2019

    Abs URL Scholar (0)

    Questionnaire used by the Cornell Criminal Records Panel Study. Subjects are members of the class in the law suit against the US Census Bureau. The purpose of this research is to learn more about the errors in background screening and understand how criminal records impact employment opportunities.

22. metajelo: A metadata package for journals to support external linked objects

    Carl Lagoze and Lars Vilhuber

    Labor Dynamics Institute, Document, 2019

    URL Scholar (0)
23. Outcomes report of the Cornell Node of the NSF-Census Research Network

    Lars Vilhuber and William Block

    Jan 2019

    Abs URL Scholar (0)

    Description and List of Outcomes of the Cornell node of the NSF-Census Research Network.

24. Disclosure Limitation and Confidentiality Protection in Linked Data

    John M. Abowd, Ian M. Schmutte, and Lars Vilhuber

    Labor Dynamics Institute, Cornell University, Document 47, Jan 2018

    Abs URL Scholar (7)

    Confidentiality protection for linked administrative data is a combination of access modalities and statistical disclosure limitation. We review traditional statistical disclosure limitation methods and newer methods based on synthetic data, input noise infusion and formal privacy. We discuss how these methods are integrated with access modalities by providing three detailed examples. The first example is the linkages in the Health and Retirement Study to Social Security Administration data. The second example is the linkage of the Survey of Income and Program Participation to administrative data from the Internal Revenue Service and the Social Security Administration. The third example is the Longitudinal Employer-Household Dynamics data, which links state unemployment insurance records for workers and firms to a wide variety of censuses and surveys at the U.S. Census Bureau. For examples, we discuss access modalities, disclosure limitation methods, the effectiveness of those methods, and the resulting analytical validity. The final sections discuss recent advances in access modalities for linked administrative data.

25. The Reproducibility of Economics Research: A Case Study

    Hautahi Kingi, Flavio Stanchi, Lars Vilhuber, and 1 more author

    2018

    URL Scholar (8)
26. LEHD infrastructure S2014 files in the FSRDC

    Lars Vilhuber

    Center for Economic Studies, U.S. Census Bureau, Working Papers 18-27, May 2018

    Abs URL

    The Longitudinal Employer-Household Dynamics (LEHD) Program at the U.S. Census Bureau, with the support of several national research agencies, maintains a set of infrastructure files using administrative data provided by state agencies, enhanced with information from other administrative data sources, demographic and economic (business) surveys and censuses. The LEHD Infrastructure Files provide a detailed and comprehensive picture of workers, employers, and their interaction in the U.S. economy. This document describes the structure and content of the 2014 Snapshot of the LEHD Infrastructure files as they are made available in the Census Bureau’s secure and restricted-access Research Data Center network. The document attempts to provide a comprehensive description of all researcher-accessible files, of their creation, and of any modifications made to the files to facilitate researcher access.

27. Issues with Unpaywall in Economics

    Lars Vilhuber

    Labor Dynamics Institute, Oct 2018

    URL
28. Reproducibility and replicability in economics

    Lars Vilhuber

    National Academies of Sciences, Engineering, and Medicine, Commissioned Paper, 2018

    URL
29. Codebook for the SIPP Synthetic Beta 7.0 (PDF version)

    Lori B. Reeder, Jordan C. Stanley, and Lars Vilhuber

    Cornell Institute for Social and Economic Research and Labor Dynamics Institute. Cornell University, Codebook V20181102b-pdf, Nov 2018

    DOI URL OA? Dimensions (0)
30. Cornell Project for Record Assistance Questionnaire - B-filers

    Linda Barrington, Esta R. Bigler, Hassan Enayati, and 3 more authors

    May 2017

    Abs URL Scholar (1)

    Questionnaire used by the Cornell Project for Records Assistance, part of the Cornell University ILR School (Industrial and Labor Relations), to collect information about respondents, their work history, their involvement with the criminal justice system, if any, and their choice of remedy under the settlement of the Gonzalez et al v. Pritzker case. These respondents had chosen not to receive remedy, information was collected to understand their choice.

31. Cornell Project for Record Assistance Questionnaire - A-filers

    Linda Barrington, Esta R. Bigler, Hassan Enayati, and 3 more authors

    May 2017

    Abs URL Scholar (1)

    Questionnaire used by the Cornell Project for Records Assistance, part of the Cornell University ILR School (Industrial and Labor Relations), to collect information about respondents, their work history, their involvement with the criminal justice system, if any, and their choice of remedy under the settlement of the Gonzalez et al v. Pritzker case. The responses were used to inform support and remedy provided to respondents.

32. Cornell Project for Records Assistance Questionnaire - with routing

    Linda Barrington, Esta R. Bigler, Hassan Enayati, and 3 more authors

    May 2017

    Abs DOI URL OA? Dimensions (0)

    Questionnaire used by the Cornell Project for Records Assistance, part of the Cornell University ILR School (Industrial and Labor Relations), to collect information about respondents, their work history, their involvement with the criminal justice system, if any, and their choice of remedy under the settlement of the Gonzalez et al v. Pritzker case. This version contains routing information.

33. Understanding the effect of procedural justice on psychological distress

    Julie Cloutier, Lars Vilhuber, Denis Harrisson, and 1 more author

    Labor Dynamics Institute, Cornell University, Document 35, 2017

    Abs URL

    Studies on the effect of procedural justice on psychological distress present conflicting results. Drawing on instrumental and relational perspectives of justice, we test the hypothesis that the perception of procedural justice influences the level of workers’ psychological distress. Using a number of validated instruments to collected data from 659 workers in three call centers, we use OLS regressions and Hayes’ PROCESS tool to show that the perception of procedural justice has a direct, unique, and independent effect on psychological distress. The perception of procedural justice has no instrumental role, the key mechanism being the relational role, suggesting that perceived injustice influences psychological distress because it threatens self-esteem. Distributive justice perceptions (recognition, promotions, job security) are not associated with psychological distress, calling into question Siegrist’s model. Our findings suggest that perceived procedural justice provides workers better evidence of the extent to which they are valued and appreciated members of their organizations than do perceptions of distributive justice. The results highlight the greater need for workers to be valued and appreciated for who they are (consideration and esteem), rather than for what they do for their organization (distributive justice of rewards).

34. Two Perspectives on Commuting: A Comparison of Home to Work Flows Across Job-Linked Survey and Administrative Files.

    Andrew S. Green, Mark J. Kutzbach, and and Lars Vilhuber

    U.S. Census Bureau Center for Economic Studies Discussion, Paper, 2017

    URL Scholar (9)
35. Recalculating - how uncertainty in local labor market definitions affects empirical findings

    Andrew Foote, Mark J. Kutzbach, and Lars Vilhuber

    NSF Census Research Network - NCRN-Cornell, Preprint 1813:52649, 2017

    Abs URL

    This paper evaluates the use of commuting zones as a local labor market definition. We revisit Tolbert and Sizer (1996) and demonstrate the sensitivity of definitions to two features of the methodology. We show how these features impact empirical estimates using a well-known application of commuting zones. We conclude with advice to researchers using commuting zones on how to demonstrate the robustness of empirical findings to uncertainty in definitions. The analysis, conclusions, and opinions expressed herein are those of the author(s) alone and do not necessarily represent the views of the U.S. Census Bureau or the Federal Deposit Insurance Corporation. All results have been reviewed to ensure that no confidential information is disclosed, and no confidential data was used in this paper. This document is released to inform interested parties of ongoing research and to encourage discussion of work in progress. Much of the work developing this paper occurred while Mark Kutzbach was an employee of the U.S. Census Bureau.

36. Utility Cost of Formal Privacy for Releasing National Employer-Employee Statistics

    Samuel Haney, Ashwin Machanavajjhala, John M. Abowd, and 3 more authors

    Labor Dynamics Institute, Cornell University, Document 36, 2017

    Abs URL Scholar (100)

    National statistical agencies around the world publish tabular summaries based on combined employer-employee (ER-EE) data. The privacy of both individuals and business establishments that feature in these data are protected by law in most countries. These data are currently released using a variety of statistical disclosure limitation (SDL) techniques that do not reveal the exact characteristics of particular employers and employees, but lack provable privacy guarantees limiting inferential disclosures. In this work, we present novel algorithms for releasing tabular summaries of linked ER-EE data with formal, provable guarantees of privacy. We show that state-of-the-art differentially private algorithms add too much noise for the output to be useful. Instead, we identify the privacy requirements mandated by current interpretations of the relevant laws, and formalize them using the Pufferfish framework. We then develop new privacy definitions that are customized to ER-EE data and satisfy the statutory privacy requirements. We implement the experiments in this paper on production data gathered by the U.S. Census Bureau. An empirical evaluation of utility for these data shows that for reasonable values of the privacy-loss parameter {}epsilon}geq 1, the additive error introduced by our provably private algorithms is comparable, and in some cases better, than the error introduced by existing SDL techniques that have no provable privacy guarantees. For some complex queries currently published, however, our algorithms do not have utility comparable to the existing traditional SDL algorithms. Those queries are fodder for future research.

37. Effects of a government-academic partnership: Has the NSF-census bureau research network helped improve the U.S. statistical system?

    Daniel H. Weinberg, John M. Abowd, Robert F. Belli, and 13 more authors

    Center for Economic Studies, U.S. Census Bureau, Working Papers 17-59r, Jan 2017

    Abs URL

    The National Science Foundation-Census Bureau Research Network (NCRN) was established in 2011 to create interdisciplinary research nodes on methodological questions of interest and significance to the broader research community and to the Federal Statistical System (FSS), particularly the Census Bureau. The activities to date have covered both fundamental and applied statistical research and have focused at least in part on the training of current and future generations of researchers in skills of relevance to surveys and alternative measurement of economic units, households, and persons. This paper discusses some of the key research findings of the eight nodes, organized into six topics: (1) Improving census and survey data collection methods; (2) Using alternative sources of data; (3) Protecting privacy and confidentiality by improving disclosure avoidance; (4) Using spatial and spatio-temporal statistical modeling to improve estimates; (5) Assessing data cost and quality tradeoffs; and (6) Combining information from multiple sources. It also reports on collaborations across nodes and with federal agencies, new software developed, and educational activities and outcomes. The paper concludes with an evaluation of the ability of the FSS to apply the NCRN’s research outcomes and suggests some next steps, as well as the implications of this research-network model for future federal government renewal initiatives.

38. Total error and variability measures with integrated disclosure limitation for quarterly workforce indicators and LEHD origin destination employment statistics in on the map

    Kevin L. McKinney, Andrew S. Green, Lars Vilhuber, and 1 more author

    Center for Economic Studies, U.S. Census Bureau, Working Papers 17-71, Jan 2017

    Abs URL

    We report results from the rst comprehensive total quality evaluation of five major indicators in the U.S. Census Bureau’s Longitudinal Employer-Household Dynamics (LEHD) Program Quarterly Workforce Indicators (QWI): total employment, beginning-of-quarter employment, full-quarter employment, total payroll, and average monthly earnings of full-quarter employees. Beginning-of-quarter employment is also the main tabulation variable in the LEHD Origin-Destination Employment Statistics (LODES) workplace reports as displayed in OnTheMap (OTM). The evaluation is conducted by generating multiple threads of the edit and imputation models used in the LEHD Infrastructure File System. These threads conform to the Rubin (1987) multiple imputation model, with each thread or implicate being the output of formal probability models that address coverage, edit, and imputation errors. Design-based sampling variability and nite population corrections are also included in the evaluation. We derive special formulas for the Rubin total variability and its components that are consistent with the disclosure avoidance system used for QWI and LODES/OTM workplace reports. These formulas allow us to publish the complete set of detailed total quality measures for QWI and LODES. The analysis reveals that the five publication variables under study are estimated very accurately for tabulations involving at least 10 jobs. Tabulations involving three to nine jobs have quality in the range generally deemed acceptable. Tabulations involving zero, one or two jobs, which are generally suppressed in the QWI and synthesized in LODES, have substantial total variability but their publication in LODES allows the formation of larger custom aggregations, which will in general have the accuracy estimated for tabulations in the QWI based on a similar number of workers.

39. Making Confidential Data Part of Reproducible Research

    Lars Vilhuber and Carl Lagoze

    Labor Dynamics Institute, Cornell University, Document 41, 2017

    URL Scholar (1)
40. Proceedings from the 2017 Cornell-Census-NSF-Sloan Workshop on Practical Privacy

    Lars Vilhuber and Ian Schmutte

    Labor Dynamics Institute, Cornell University, Document 43, 2017

    Abs URL Scholar (0)

    These proceedings report on a workshop hosted at the U.S. Census Bureau on May 8, 2017. Our purpose was to gather experts from various backgrounds together to continue discussing the development of formal privacy systems for Census Bureau data products. This workshop was a successor to a previous workshop held in October 2016 (Vilhuber & Schmutte 2017). At our prior workshop, we hosted computer scientists, survey statisticians, and economists, all of whom were experts in data privacy. At that time we discussed the practical implementation of cutting-edge methods for publishing data with formal, provable privacy guarantees, with a focus on applications to Census Bureau data products. The teams developing those applications were just starting out when our first workshop took place, and we spent our time brainstorming solutions to the various problems researchers were encountering, or anticipated encountering. For these cutting-edge formal privacy models, there had been very little effort in the academic literature to apply those methods in real-world settings with large, messy data. We therefore brought together an expanded group of specialists from academia and government who could shed light on technical challenges, subject matter challenges and address how data users might react to changes in data availability and publishing standards. In May 2017, we organized a follow-up workshop, which these proceedings report on. We reviewed progress made in four different areas. The four topics discussed as part of the workshop were 1. the 2020 Decennial Census; 2. the American Community Survey (ACS); 3. the 2017 Economic Census; 4. measuring the demand for privacy and for data quality. As in our earlier workshop, our goals were to 1. Discuss the specific challenges that have arisen in ongoing efforts to apply formal privacy models to Census data products by drawing together expertise of academic and governmental researchers; 2. Produce short written memos that summarize concrete suggestions for practical applications to specific Census Bureau priority areas.

41. Proceedings from the Synthetic LBD International Seminar

    Lars Vilhuber, Saki Kinney, and Ian Schmutte

    Labor Dynamics Institute, Cornell University, Document 44, 2017

    Abs URL Scholar (0)

    On May 9, 2017, we hosted a seminar to discuss the conditions necessary to implement the SynLBD approach with interested parties, with the goal of providing a straightforward toolkit to implement the same procedure on other data. The proceedings summarize the discussions during the workshop.

42. Proceedings from the 2016 NSF-Sloan Workshop on Practical Privacy

    Lars Vilhuber and Ian Schmutte

    Labor Dynamics Institute, Cornell University, Document 33, 2017

    URL Scholar (0)
43. Using partially synthetic microdata to protect sensitive cells in business statistics

    Javier Miranda and Lars Vilhuber

    NSF Census Research Network - NCRN-Cornell 1813:42339, 2016

    Abs URL

    We describe and analyze a method that blends records from both observed and synthetic microdata into public-use tabulations on establishment statistics. The resulting tables use synthetic data only in potentially sensitive cells. We describe different algorithms, and present preliminary results when applied to the Census Bureau’s Business Dynamics Statistics and Synthetic Longitudinal Business Database, highlighting accuracy and protection afforded by the method when compared to existing public-use tabulations (with suppressions).

44. Synthetic establishment microdata around the world

    Lars Vilhuber, John A. Abowd, and Jerome P. Reiter

    NSF Census Research Network - NCRN-Cornell 1813:42340, 2016

    tex.owner: vilhuber tex.timestamp: 2014.03.24

    Abs URL

    In contrast to the many public-use microdata samples available for individual and household data from many statistical agencies around the world, there are virtually no establishment or firm microdata available. In large part, this difficulty in providing access to business micro data is due to the skewed and sparse distributions that characterize business data. Synthetic data are simulated data generated from statistical models. We organized sessions at the 2015 World Statistical Congress and the 2015 Joint Statistical Meetings, highlighting work on synthetic establishment microdata. This overview situates those papers, published in this issue, within the broader literature.

45. Looking back on three years of using the Synthetic LBD beta

    Javier Miranda and Lars Vilhuber

    Center for Economic Studies, U.S. Census Bureau, Working Papers 14-11, Feb 2014

    tex.owner: vilhuber tex.timestamp: 2014.03.24

    Abs URL

    Distributions of business data are typically much more skewed than those for household or individual data and public knowledge of the underlying units is greater. As a results, national statistical offices (NSOs) rarely release establishment or firm-level business microdata due to the risk to respondent confidentiality. One potential approach for overcoming these risks is to release synthetic data where the establishment data are simulated from statistical models designed to mimic the distributions of the real underlying microdata. The US Census Bureau?s Center for Economic Studies in collaboration with Duke University, the National Institute of Statistical Sciences, and Cornell University made available a synthetic public use file for the Longitudinal Business Database (LBD) comprising more than 20 million records for all business establishment with paid employees dating back to 1976. The resulting product, dubbed the SynLBD, was released in 2010 and is the first-ever comprehensive business microdata set publicly released in the United States including data on establishments employment and payroll, birth and death years, and industrial classification. This pa- per documents the scope of projects that have requested and used the SynLBD.

46. A First Step Towards A German SynLBD: Constructing A German Longitudinal Business Database

    Jörg Drechsler and Lars Vilhuber

    Center for Economic Studies, U.S. Census Bureau, Working Papers 14-13, Feb 2014

    Abs URL

    One major criticism against the use of synthetic data has been that the efforts necessary to generate useful synthetic data are so in- tense that many statistical agencies cannot afford them. We argue many lessons in this evolving field have been learned in the early years of synthetic data generation, and can be used in the development of new synthetic data products, considerably reducing the required in- vestments. The final goal of the project described in this paper will be to evaluate whether synthetic data algorithms developed in the U.S. to generate a synthetic version of the Longitudinal Business Database (LBD) can easily be transferred to generate a similar data product for other countries. We construct a German data product with infor- mation comparable to the LBD - the German Longitudinal Business Database (GLBD) - that is generated from different administrative sources at the Institute for Employment Research, Germany. In a fu- ture step, the algorithms developed for the synthesis of the LBD will be applied to the GLBD. Extensive evaluations will illustrate whether the algorithms provide useful synthetic data without further adjustment. The ultimate goal of the project is to provide access to multiple synthetic datasets similar to the SynLBD at Cornell to enable comparative studies between countries. The Synthetic GLBD is a first step towards that goal.

47. LEHD Infrastructure files in the Census RDC - Overview

    Lars Vilhuber and Kevin McKinney

    Center for Economic Studies, U.S. Census Bureau 14-26, Jun 2014

    Abs URL Scholar (27)

    The Longitudinal Employer-Household Dynamics (LEHD) Program at the U.S. Census Bureau, with the support of several national research agencies, maintains a set of infrastructure files using administrative data provided by state agencies, enhanced with information from other administrative data sources, demographic and economic (business) surveys and censuses. The LEHD Infrastructure Files provide a detailed and comprehensive picture of workers, employers, and their interaction in the U.S. economy. This document describes the structure and content of the 2011 Snapshot of the LEHD Infrastructure files as they are made available in the Census Bureaus secure and restricted-access Research Data Center network. The document attempts to provide a comprehensive description of all researcher-accessible files, of their creation, and of any modifcations made to the files to facilitate researcher access.

48. Estimation de la contribution de la réallocation de la main-d’oeuvre à la croissance de la productivité au Canada

    Charles Bérubé, Benoit Dostie, and Lars Vilhuber

    Centre sur la productivité et la prospérité, HEC Montréal, 2013

    Abs URL

    In this report, we estimate the contribution of labour reallocation to productivity growth in the Canadian manufacturing sector. We find that most of productivity growth comes from within firm improvements, leaving a limited role for labour reallocation. Still, we also find that the importance of labour reallocation increase over time. This is both due to increasing net-entry and inter-firm effects. These effects are much more important post 2000 than in the 1990s. We also find that lost production from exiting firms is now most likely replaced by production from existing firms, while previously, it was more likely to be replaced by production from new firms. (French only)

49. Methods for Protecting the Confidentiality of Firm-Level Data: Issues and Solutions

    Lars Vilhuber

    Labor Dynamics Institute 19, Mar 2013

    Abs URL Scholar (5)

    This report will provide an overview of methods used by statistical agencies to encourage, support, and enhance research access to data for the purpose of generating new knowledge. Quite a few reports and scientific articles have addressed the issue before, and we will be highly indebted to that literature. To a summary of that literature, we hope to provide some recent developments and experiences derived from a decade of working with systems that increase access as both researchers as well as data providers. The report will focus on the data provided by statistical agencies, but it should be understood that government agencies other than a National Statistical Office (NSO) may acquire that function. While excluding the legal background limiting or permitting such data collection and provision, we will highlight some alternate sources and methods, prior to concluding.

50. Dynamically Consistent Noise Infusion and Partially Synthetic Data as Confidentiality Protection Measures for Related Time Series

    John M Abowd, R Kaj Kaj Gittings, Kevin L McKinney, and 3 more authors

    U.S. Census Bureau, Center for Economic Studies 12-13, Oct 2012

    Abs DOI URL Scholar (3) Dimensions (15)

    The Census Bureau’s Quarterly Workforce Indicators (QWI) provide detailed quarterly statistics on employment measures such as worker and job flows, tabulated by

51. New york state disability and employment status report, 2011

    Sarah Von Schrader, William Erickson, Thomas Golden, and 1 more author

    Cornell University, Employment and Disability Institute, Report on behalf of New York Makes Work Pay Comprehensive Employment System Medicaid Infrastructure Grant, 2011

    URL
52. LEHD Infrastructure files in the Census RDC-Overview S2008

    Kevin L. McKinney and Lars Vilhuber

    U.S. Census Bureau, Working Papers 11-43, 2011

    URL Scholar (27)
53. LEHD Infrastructure Files in the Census RDC: Overview of S2004 Snapshot

    Kevin L McKinney and Lars Vilhuber

    U.S. Census Bureau, Working Papers 11-13, Apr 2011

    Abs URL Scholar (10)

    The Longitudinal Employer-Household Dynamics (LEHD) Program at the U.S. Census Bureau, with the support of several national research agencies, has built a set o

54. National estimates of gross employment and job flows from the Quarterly Workforce Indicators with demographic and industry detail (with color graphs)

    John M. Abowd and Lars Vilhuber

    Center for Economic Studies, U.S. Census Bureau, Working Papers 10-11, Jun 2010

    Abs URL

    The Quarterly Workforce Indicators (QWI) are local labor market data produced and released every quarter by the United States Census Bureau. Unlike any other local labor market series produced in the U.S. or the rest of the world, the QWI measure employment flows for workers (accession and separations), jobs (creations and destructions) and earnings for demographic subgroups (age and gender), economic industry (NAICS industry groups), detailed geography (block (experimental), county, Core- Based Statistical Area, and Workforce Investment Area), and ownership (private, all) with fully interacted publication tables. The current QWI data cover 47 states, about 98% of the private workforce in those states, and about 92% of all private employment in the entire economy. State participation is sufficiently extensive to permit us to present the first national estimates constructed from these data. We focus on worker, job, and excess (churning) reallocation rates, rather than on levels of the basic variables. This permits comparison to existing series from the Job Openings and Labor Turnover Survey and the Business Employment Dynamics Series from the Bureau of Labor Statistics. The national estimates from the QWI are an important enhancement to existing series because they include demographic and industry detail for both worker and job flow data compiled from underlying micro-data that have been integrated at the job and establishment levels by the Longitudinal Employer-Household Dynamics Program at the Census Bureau. The estimates presented herein were compiled exclusively from public-use data series and are available for download.

55. New york state disability and employment status report, 2009

    Sarah Von Schrader, William Erickson, Lars Vilhuber, and 1 more author

    Cornell University, Employment and Disability Institute, Report on behalf of New York Makes Work Pay Comprehensive Employment System Medicaid Infrastructure Grant, 2010

    URL
56. Measuring firm-level displacement events with administrative data

    Lars Vilhuber

    Workshop on Measurement Error in Administrative Data, 2010
57. Adjusting imperfect data: Overview and case studies

    Lars Vilhuber

    NBER, Working paper 12977, 2007

    DOI URL Dimensions (0)
58. Confidentiality Protection in the Census Bureau’s Quarterly Workforce Indicators

    John M. Abowd, Bryce E. Stephens, and Lars Vilhuber

    U.S. Census Bureau, LEHD and Cornell University, presented at the Joint Statistical Meetings 2005, Minneapolis, MN. 2006-02, 2005

    URL
59. Adjusting imperfect data: Overview and case studies

    Lars Vilhuber

    LEHD, Technical paper TP-2004-05, 2004

    URL
60. Abandoning the sinking ship: The composition of worker flows prior to displacement

    Paul A. Lengermann and Lars Vilhuber

    LEHD, U.S. Census Bureau, Technical paper TP-2002-11, 2002

    URL
61. The creation of the employment dynamics estimates

    John M. Abowd, Paul A. Lengermann, and Lars Vilhuber

    LEHD, U.S. Census Bureau, Technical paper TP-2002-13, 2002

    URL
62. The sensitivity of economic statistics to coding errors in personal identifiers

    John M. Abowd and Lars Vilhuber

    LEHD, U.S. Census Bureau, Technical paper TP-2002-17, 2002

    URL
63. Displaced workers, early leavers, and re-employment wages

    Audra Bowlus and Lars Vilhuber

    LEHD, U.S. Census Bureau, Technical paper TP-2002-18, 2002

    URL
64. Escaping poverty for low-wage workers: The role of employer characteristics and changes

    Harry J. Holzer, Julia I. Lane, Lars Vilhuber, and 2 more authors

    LEHD, U.S. Census Bureau, Technical paper TP-2001-02, 2001

    URL
65. Longitudinal analysis of SSN response on SIPP 1990-1993 panels

    Lars Vilhuber and Robert Pedace

    LEHD, U.S. Census Bureau, Technical paper TP-2000-01, 2000

    URL
66. Continuous Training and sectoral mobility in Germany

    Lars Vilhuber

    CIRANO, Scientific Series 99s-03, 1999

    Abs

    This article studies mobility patterns of German workers in light of a model of sector-specific human capital. Furthermore, I employ and describe little-used data on continuous on-the-job training occuring after apprenticeships. Results are presented describing the incidence and duration of continuous training. Continuous training is quite common, depite the high incidence of apprenticeships which precedes this part of a worker’s career. Most previous studies have only distinguished between firm-specific and general human capital, generally concluding that training was general. Inconsistent with those conclusions, I show that German men are more likely to find a job within the same sector if they have received continuous training in that sector. These results are similar to results obtained for young U.S. workers, and suggest that sector-specific capital is an important feature of very different labor markets. Furthermore, the results suggest that the observed effect of training on mobility is sensitive to the state of the business cycle, indicating a more complex interaction between supply and demand that most theoretical models allow for.

67. Sector-specific on-the-job training: Evidence from U.S. data

    Lars Vilhuber

    CIRANO, Scientific Series 97s-42, 1997

    Abs

    Using data from the National Longitudinal Survey of Youth (NLSY), we re-examine the effect of formal on-the-job training on mobility patterns of young American workers. By employing parametric duration models, we evaluate the economic impact of training on productive time with an employer. Confirming previous studies, we find a positive and statistically significant impact of formal on-the-job training on tenure with the employer providing the training. However, expected duration net of the time spent in the training program is generally not significantly increased. We proceed to document and analyze intra-sectoral and cross-sectoral mobility patterns in order to infer whether training provides firm-specific, industry-specific, or general human capital. The econometric analysis rejects a sequential model of job separation in favor of a competing risks specification. We find significant evidence for the industry-specificity of training. The probability of sectoral mobility upon job separation decreases with training received in the current industry, whether with the last employer or previous employers, and employment attachment increases with on-the-job training. These results are robust to a number of variations on the base model.

68. Wage flexibility and contract structure in Germany

    Lars Vilhuber

    CIRANO, Scientific Series 96s-28, 1996

    Abs

    In this paper, we look at how labor market conditions at different points during the tenure of individuals with firms are correlated with current earnings. Using data from the German Socioeconomic Panel on individuals for the period 1984 to 1994, we find that both the contemporaneous unemployment rate and prior values of the unemployment rate are significantly correlated with current earnings, contrary to results for the American labor market. We interpret this result as evidence that German unions do in fact bargain over both wages and employment, but that the models of individualistic contracts, such as the implicit contract model, may explain some of the observed wage drift and longer-term wage movements reasonably well. Furthermore, we explore the heterogeneity of contracts over a variety of worker and job characteristics. In particular, we find evidence that contracts differ across industries and across firm size. Workers of large firms are remarkably more insulated from the job market than workers for any other type of firm, indicating the importance of internal job markets.

Conference Papers (25)

1. Report of the AEA Data Editor

   Lars Vilhuber and Jack Cavanagh

   In AEA Papers and Proceedings, May 2025

   tex.ids= vilhuber2025

   DOI URL Scholar (0) Dimensions (0)
2. Report of the AEA Data Editor

   Lars Vilhuber

   In AEA Papers and Proceedings, May 2024

   DOI URL Scholar (0) Dimensions (0)
3. Report of the AEA Data Editor

   Lars Vilhuber

   In AEA Papers and Proceedings, May 2023

   DOI URL Scholar (0) Dimensions (3)
4. Report by the AEA Data Editor

   Lars Vilhuber

   In AEA Papers and Proceedings, May 2022

   DOI Scholar (0) Dimensions (7)
5. Report by the AEA Data Editor

   Lars Vilhuber

   In AEA Papers and Proceedings, May 2021

   DOI URL Scholar (0) Dimensions (9)
6. Report by the AEA Data Editor

   Lars Vilhuber, James Turitto, and Keesler Welch

   In AEA Papers and Proceedings, May 2020

   DOI Scholar (0) Dimensions (13)
7. Report by the AEA Data Editor

   Lars Vilhuber

   In AEA Papers and Proceedings, May 2019

   DOI URL Scholar (0) Dimensions (22)
8. Why the Economics Profession Must Actively Participate in the Privacy Protection Debate

   John M. Abowd, Ian M. Schmutte, William N. Sexton, and 1 more author

   In AEA Papers and Proceedings, May 2019

   Abs DOI URL OA? Scholar (14) Dimensions (6)

   When Google or the U.S. Census Bureau publish detailed statistics on browsing habits or neighborhood characteristics, some privacy is lost for everybody while supplying public information. To date, economists have not focused on the privacy loss inherent in data publication. In their stead, these issues have been advanced almost exclusively by computer scientists who are primarily interested in technical problems associated with protecting privacy. Economists should join the discussion, first, to determine where to balance privacy protection against data quality; a social choice problem. Furthermore, economists must ensure new privacy models preserve the validity of public data for economic research.

9. Making Confidential Data Part of Reproducible Research

   Lars Vilhuber

   In Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop, 2019

   Abs DOI URL Scholar (1) Dimensions (1)

   In 2014 the National Science Foundation (NSF) provided support to the National Academies of Sciences, Engineering, and Medicine for a series of Forums on Open Science in response to a government-wide directive to support increased public access to the results of research funded by the federal government. However, the breadth of the work resulting from the series precluded a focus on any specific topic or discussion about how to improve public access. Thus, the main goal of the Workshop on Transparency and Reproducibility in Federal Statistics was to develop some understanding of what principles and practices are, or would be, supportive of making federal statistics more understandable and reviewable, both by agency staff and the public. This publication summarizes the presentations and discussions from the workshop.

10. Synthetic data via quantile regression for heavy-tailed and heteroskedastic data

    Michelle Pistner, Aleksandra Slavković, and Lars Vilhuber

    In Privacy in statistical databases, 2018

    See https://github.com/labordynamicsinstitute/replication_qr_synthetic for replication code.

    DOI URL OA? Dimensions (7)
11. Utility Cost of Formal Privacy for Releasing National Employer-Employee Statistics

    Samuel Haney, Ashwin Machanavajjhala, John M. Abowd, and 3 more authors

    In Proceedings of the 2017 International Conference on Management of Data, 2017

    Abs DOI URL Scholar (100) Dimensions (52)

    National statistical agencies around the world publish tabular summaries based on combined employer-employee (ER-EE) data. The privacy of both individuals and business establishments that feature in these data are protected by law in most countries. These data are currently released using a variety of statistical disclosure limitation (SDL) techniques that do not reveal the exact characteristics of particular employers and employees, but lack provable privacy guarantees limiting inferential disclosures. In this work, we present novel algorithms for releasing tabular summaries of linked ER-EE data with formal, provable guarantees of privacy. We show that state-of-the-art differentially private algorithms add too much noise for the output to be useful. Instead, we identify the privacy requirements mandated by current interpretations of the relevant laws, and formalize them using the Pufferfish framework. We then develop new privacy definitions that are customized to ER-EE data and satisfy the statutory privacy requirements. We implement the experiments in this paper on production data gathered by the U.S. Census Bureau. An empirical evaluation of utility for these data shows that for reasonable values of the privacy-loss parameter {}epsilon}geq 1, the additive error introduced by our provably private algorithms is comparable, and in some cases better, than the error introduced by existing SDL techniques that have no provable privacy guarantees. For some complex queries currently published, however, our algorithms do not have utility comparable to the existing traditional SDL algorithms. Those queries are fodder for future research.

12. Proceedings from the 2016 NSF–Sloan Workshop on Practical Privacy

    Lars Vilhuber, Ian M. Schmutte, and and John M. Abowd

    In 2016 NSF–Sloan Workshop on Practical Privacy, 2017

    URL Scholar (0)
13. CED2AR: The Comprehensive Extensible Data Documentation and Access Repository.

    Carl Lagoze, Lars Vilhuber, Jeremy Williams, and 2 more authors

    In ACM/IEEE Joint Conference on Digital Libraries (JCDL 2014), 2014

    Abs DOI URL OA? Scholar (6) Dimensions (2)

    We describe the design, implementation, and deployment of the Comprehensive Extensible Data Documentation and Access Repository (CED2AR). This is a metadata repository system that allows researchers to search, browse, access, and cite confidential data and metadata through either a web-based user interface or programmatically through a search API, all the while re-reusing and linking to existing archive and provider generated metadata. CED2AR is distinguished from other metadata repository-based applications due to requirements that derive from its social science context. These include the need to cloak confidential data and metadata and manage complex provenance chains.

14. Using partially synthetic data to replace suppression in the business dynamics statistics: Early results

    Javier Miranda and Lars Vilhuber

    In Privacy in statistical databases, 2014

    Abs DOI URL OA? Dimensions (3)

    The Business Dynamics Statistics is a product of the U.S. Census Bureau that provides measures of business openings and closings, and job creation and destruction, by a variety of cross-classifications (firm and establishment age and size, industrial sector, and geography). Sensitive data are currently protected through suppression. However, as additional tabulations are being developed, at ever more detailed geographic levels, the number of suppressions increases dramatically. This paper explores the option of providing public-use data that are analytically valid and without suppressions, by leveraging synthetic data to replace observations in sensitive cells.

15. Synthetic longitudinal business databases for international comparisons

    Jörg Drechsler and Lars Vilhuber

    In Privacy in statistical databases, 2014

    Abs DOI URL OA? Dimensions (2)

    International comparison studies on economic activity are often hampered by the fact that access to business microdata is very limited on an international level. A recently launched project tries to overcome these limitations by improving access to Business Censuses from multiple countries based on synthetic data. Starting from the synthetic version of the longitudinally edited version of the U.S. Business Register (the Longitudinal Business Database, LBD), the idea is to create similar data products in other countries by applying the synthesis methodology developed for the LBD to generate synthetic replicates that could be distributed without confidentiality concerns. In this paper we present some first results of this project based on German business data collected at the Institute for Employment Research.

16. CED²AR: The Comprehensive Extensible Data Documentation and Access Repository

    Carl Lagoze, Lars Vilhuber, Jeremy Williams, and 2 more authors

    In ACM/IEEE Joint Conference on Digital Libraries (JCDL 2014), Sep 2014

    Presented at the ACM/IEEE Joint Conference on Digital Libraries (JCDL 2014)

    Abs DOI URL OA? Dimensions (2)

    Social science researchers increasingly make use of data that is confidential because it contains linkages to the identities of people, corporations, etc. The value of this data lies in the ability to join the identifiable entities with external data such as genome data, geospatial information, and the like. However, the confidentiality of this data is a barrier to its utility and curation, making it difficult to fulfill US federal data management mandates and interfering with basic scholarly practices such as validation and reuse of existing results. We describe the complexity of the relationships among data that span a public and private divide. We then describe our work on the CED2AR prototype, a first step in providing researchers with a tool that spans this divide and makes it possible for them to search, access, and cite that data.

17. Replicating the Synthetic LBD with German establishment data

    Jörg Drechsler and Lars Vilhuber

    In Proceedings 59th ISI world statistics congress, 25-30 august 2013, hong kong (session STS062), 2013

    URL
18. Encoding Provenance Metadata for Social Science Datasets.

    Carl Lagoze, Jeremy Willliams, and Lars Vilhuber

    In Metadata and Semantics Research. Communications in Computer and Information Science, 2013

    DOI OA? Scholar (10) Dimensions (7)
19. Encoding provenance of social science data: Integrating PROV with DDI

    Carl Lagoze, William C. Block, Jeremy Williams, and 1 more author

    In 5th annual european DDI user conference, 2013

    Abs URL

    Provenance is a key component of evaluating the integrity and reusability of data for scholarship. While recording and providing access provenance has always been important, it is even more critical in the web environment in which data from distributed sources and of varying integrity can be combined and derived. The PROV model, developed under the auspices of the W3C, is a foundation for semantically-rich, interoperable, and web-compatible provenance metadata. We report on the results of our experimentation with integrating the PROV model into the DDI metadata for a complex, but characteristic, example social science data. We also present some preliminary thinking on how to visualize those graphs in the user interface.

20. Encoding Provenance of Social Science Data: Integrating PROV with DDI

    Carl Lagoze, William C. Block, Jeremy Williams, and 1 more author

    In 5th Annual European DDI User Conference, 2013

    Abs DOI URL OA? Dimensions (0)

    Provenance is a key component of evaluating the integrity and reusability of data for scholarship. While recording and providing access provenance has always been important, it is even more critical in the web environment in which data from distributed sources and of varying integrity can be combined and derived. The PROV model, developed under the auspices of the W3C, is a foundation for semantically-rich, interoperable, and web-compatible provenance metadata. We report on the results of our experimentation with integrating the PROV model into the DDI metadata for a complex, but characteristic, example social science data. We also present some preliminary thinking on how to visualize those graphs in the user interface.

21. A Proposed Solution to the Archiving and Curation of Confidential Scientific Inputs

    J.M. Abowd, Lars Vilhuber, and William C. Block

    In Privacy in Statistical Databases, 2012

    Abs DOI URL OA? Scholar (11) Dimensions (3)

    We develop the core of a method for solving the data archive and curation problem that confronts the custodians of restricted-access research data and the scientific users of such data. Our solution recognizes the dual protections afforded by physical security and access limitation protocols. It is based on extensible tools and can be easily incorporated into existing instructional materials.

22. Using linked employer-employee data to investigate the speed of adjustment in downsizing firms in Canada and the US

    Benoit Dostie, Kevin L. McKinney, and Lars Vilhuber

    In International census research data center conference, Oct 2009

    Abs

    When firms are faced with a demand shock, adjustment can take many forms. Firms can adjust physical capital, human capital, or both. The speed of adjustment may differ as well: costs of adjustment, the type of shock, the legal and economic enviroment all matter. In this paper, we focus on firms that downsized between 1992 and 1997, but ultimately survive, and investigate how the human capital distribution within a firm influences the speed of adjustment, \textitceteris paribus. In other words, when do firms use mass layoffs instead of attrition to adjust the level of employment. We combine worker-level wage records and measures of human capital with firm-level characteristics of the production function, and use levels and changes in these variables to characterize the choice of adjustment method and speed. Firms are described/compared up to 9 years prior to death. We also consider how workers fare after leaving downsizing firms, and analyze if observed differences in post-separation outcomes of workers provide clues to the choice of adjustment speed.

23. How protective are synthetic data

    John Abowd and Lars Vilhuber

    In Privacy in statistical database, Sep 2008

    DOI URL OA? Dimensions (77)
24. Privacy: Theory meets practice on the map

    Ashwin Machanavajjhala, Daniel Kifer, John M Abowd, and 2 more authors

    In Proceedings of the International Conference on Data Engineering, 2008

    DOI URL OA? Scholar (915) Dimensions (482)
25. Using linked employer-employee data to investigate the speed of adjustment in downsizing firms

    Kevin L. McKinney and Lars Vilhuber

    In Conference on the analysis of firms and employees (CAFE), Sep 2006

    Abs URL

    When firms are faced with a demand shock, adjustment can take many forms. Firms can adjust physical capital, human capital, or both. The speed of adjustment may differ as well: costs of adjustment, the type of shock, the legal and economic enviroment all matter. In this paper, we focus on firms that downsized between 1992 and 1997, but ultimately survive, and investigate how the human capital distribution within a firm influences the speed of adjustment, \textitceteris paribus. In other words, when do firms use mass layoffs instead of attrition to adjust the level of employment. We combine worker-level wage records and measures of human capital with firm-level characteristics of the production function, and use levels and changes in these variables to characterize the choice of adjustment method and speed. Firms are described/compared up to 9 years prior to death. We also consider how workers fare after leaving downsizing firms, and analyze if observed differences in post-separation outcomes of workers provide clues to the choice of adjustment speed.

Book Chapters (12)

1. Improving Privacy for Respondents in Randomized Controlled Trials: A Differential Privacy Approach

   Soumya Mukherjee, Aratrika Mustafi, Aleksandra Slavković, and 1 more author

   In Data Privacy Protection and the Conduct of Applied Research: Methods, Approaches and New Findings, 2026

   Longer version: https://doi.org/10.48550/arXiv.2309.14581 (v1). Corrected version: https://doi.org/10.48550/ARXIV.2309.14581 (v2).

   Abs URL Scholar (6)

   Randomized control trials, RCTs, have become a powerful tool for assessing the impact of interventions and policies in many contexts. They are considered the gold-standard for inference in the biomedical fields and in many social sciences. Researchers have published an increasing number of studies that rely on RCTs for at least part of the inference, and these studies typically include the response data collected, de-identified and sometimes protected through traditional disclosure limitation methods. In this paper, we empirically assess the impact of strong privacy-preservation methodology (with }ac{DP} guarantees), on published analyses from RCTs, leveraging the availability of replication packages (research compendia) in economics and policy analysis. We provide simulations studies and demonstrate how we can replicate the analysis in a published economics article on privacy-protected data under various parametrizations. We find that relatively straightforward DP-based methods allow for inference-valid protection of the published data, though computational issues may limit more complex analyses from using these methods. The results have applicability to researchers wishing to share RCT data, especially in the context of low- and middle-income countries, with strong privacy protection.

2. Using Containers for Analysis Validation at Scale

   Lars Vilhuber

   In Data Privacy Protection and the Conduct of Applied Research: Methods, Approaches and New Findings, 2026

   For expanded version, see https://doi.org/10.1162/99608f92.4d1853ce
3. Protecting Confidential Data through Non-Statistical Methods

   Lars Vilhuber

   In Handbook of sharing confidential data: differential privacy, secure multiparty computation, and synthetic data, 2024

   Abs Scholar (2)

   "Statistical agencies, research organizations, companies, and other data stewards that seek to share data with the public face a challenging dilemma. They need to protect the privacy and confidentiality of data subjects’ and their attributes while providing data products that are useful for their intended purposes. In an age when information on data subjects is available from a wide range of data sources, as are the computational resources to obtain that information, this challenge is increasingly difficult. The Handbook of Sharing Confidential Data helps data stewards understand how tools from the data confidentiality literature-specifically, synthetic data, formal privacy, and secure computation-can be used to manage trade-offs in disclosure risk and data usefulness"–

4. Disclosure Limitation and Confidentiality Protection in Linked Data

   John M. Abowd, Ian M. Schmutte, and Lars Vilhuber

   In Administrative Records for Survey Methodology, Apr 2021

   Abs URL Scholar (7)

   Confidentiality protection for linked administrative data is a combination of access modalities and statistical disclosure limitation. We review traditional statistical disclosure limitation methods and newer methods based on synthetic data, input noise infusion and formal privacy. We discuss how these methods are integrated with access modalities by providing three detailed examples. The first example is the linkages in the Health and Retirement Study to Social Security Administration data. The second example is the linkage of the Survey of Income and Program Participation to administrative data from the Internal Revenue Service and the Social Security Administration. The third example is the Longitudinal Employer-Household Dynamics data, which links state unemployment insurance records for workers and firms to a wide variety of censuses and surveys at the U.S. Census Bureau. For examples, we discuss access modalities, disclosure limitation methods, the effectiveness of those methods, and the resulting analytical validity. The final sections discuss recent advances in access modalities for linked administrative data.

5. Using Administrative Data for Research and Evidence-Based Policy: An Introduction

   Shawn Cole, Iqbal Dhaliwal, Anja Sautmann, and 1 more author

   In Handbook on Using Administrative Data for Research and Evidence-based Policy, Jan 2021

   DOI URL Scholar (30) Dimensions (9)
6. Balancing Privacy and Data Usability: An Overview of Disclosure Avoidance Methods

   Ian M. Schmutte and Lars Vilhuber

   In Handbook on Using Administrative Data for Research and Evidence-based Policy, Jan 2021

   Abs DOI URL Scholar (5) Dimensions (9)

   The Five Safes framework (safe projects, safe people, safe settings, safe data, and safe outputs) is one way of thinking about security of different aspects of a project, and is used throughout the Handbook and in research with administrative data. Within the Five Safes framework, data providers need to create safe data that can be provided to trusted safe people for use within safe settings, as part of safe projects. Finally, any findings that are shared publicly must be safe outputs. The processes used to create safe data and safe outputs (manipulations that render data less sensitive and therefore more appropriate for public release) are generally referred to as statistical disclosure limitation (SDL). This chapter describes techniques traditionally used within the field of SDL, pointing at methods as well as metrics to assess the resultant statistical quality and sensitivity of the data, and offers technical guidance applicable to any data provider or researcher looking for practical tools to apply to their own data to reduce the risk to privacy.

7. Physically Protecting Sensitive Data

   Jim Shen and Lars Vilhuber

   In Handbook on Using Administrative Data for Research and Evidence-based Policy, Jan 2021

   Abs DOI URL Scholar (4) Dimensions (9)

   Keeping sensitive data safe relies heavily on the physical environments in which data are stored, processed, transmitted, and accessed, and from which researchers can access computers that store and process the data. However, it is also the setting that is most dependent on rapidly evolving technology. The chapter provides snapshot of the technologies available and in use as of 2020, and characterizes the technologies along a multi-dimensional scale, allowing for some comparability across methods.

8. The LEHD Infrastructure Files and the Creation of the Quarterly Workforce Indicators

   J M Abowd, B E Stephens, L Vilhuber, and 4 more authors

   In Producer Dynamics: New Evidence from Micro Data, 2009

   URL Scholar (460)
9. The link between human capital, mass layoffs, and firm deaths

   John M. Abowd, Kevin L. McKinney, and Lars Vilhuber

   In Producer dynamics: New evidence from micro data, 2009

   URL
10. Adjusting imperfect data: Overview and case studies

    Lars Vilhuber

    In The structure of wages: An international comparison, Jan 2009

    URL
11. How did universal primary education affect returns to education and labor market participation in uganda?

    Lisa Dragoset and Lars Vilhuber

    In Youth in africa’s labor market, 2008

    DOI OA? Dimensions (0)
12. Early career experiences and later career outcomes: An InternationalComparison

    David N. Margolis, Erik Plug, Véronique Simonnet, and 1 more author

    In Human capital over the life cycle - A European perspective, 2004

    Section: 5

Books (1)

1. Handbook on Using Administrative Data for Research and Evidence-based Policy

   Shawn Cole, Iqbal Dhaliwal, Anja Sautmann, and 1 more author

   Jan 2021

   Abs DOI URL Scholar (30) Dimensions (9)

   The Handbook serves as a go-to reference for researchers seeking to use administrative data and for data providers looking to make their data accessible for research. The handbook is published online under an open licensing model and freely available to all. It provides information, best practices, and case studies on how to create privacy-protected access to, handle, and analyze administrative data, with the aim of pushing the research frontier as well as informing evidence-based policy innovations.

Data & Software (27)

1. labordynamicsinstitute/metajelo-ui: v0.1.1

   Brandon Elam Barker and Lars Vilhuber

   Feb 2021

   Abs DOI URL OA? Dimensions (0)

   Targets the upcoming release of Metajelo v0.9. Updates from previous release: describe CI/build process change title of page to not be purescript-web-xpath move related identifiers to beginning in preview (metajelo-web) replace travis badge with github actions badge describe that CI builds the product using the github actions list ancillary repositories in metajelo-ui repositories with unversioned zenodo badges

2. labordynamicsinstitute/metajelo-web: v2.0.0

   Brandon Elam Barker and Lars Vilhuber

   Feb 2021

   Abs DOI URL OA? Dimensions (0)

   Targets the upcoming release of Metajelo v0.9.

3. Data for: Requesting replication materials via email

   Lars Vilhuber

   Nov 2020

   Abs DOI URL Dimensions (0)

   No description provided.

4. labordynamicsinstitute/readin_qcew_sas: A sequence of programs to readin in QCEW data from the Bureau of Labor Statistics

   Lars Vilhuber and Melissa Bjelland

   Jun 2020

   Abs DOI URL OA? Dimensions (0)

   These programs download and readin bulk data from the U.S. Bureau of Labor Statistics for the Quarterly Census of Employment and Wages data in ENB format.

5. Uncertainty in times of COVID-19: Raw survey data

   Fabian Lange, Lars Vilhuber, and Nicholas Gordon

   Labor Dynamics Institute, [data] v20200622-clean, Jul 2020

   DOI URL OA? Dimensions (0)
6. Replication code and data for: Recalculating ... How Uncertainty in Local Labor Market Definitions Affects Empirical Findings

   Andrew D. Foote, Mark J. Kutzbach, and Lars Vilhuber

   Oct 2020

   See also https://larsvilhuber.github.io/MobZ/README.html.

   DOI URL OA? Dimensions (0)
7. Applying Data Synthesis for Longitudinal Business Data across Three Countries [data and code]

   M. Jahangir Alam, Benoit Dostie, Jörg Drechsler, and 1 more author

   May 2020

   See also https://labordynamicsinstitute.github.io/SyntheticLEAP/.

   DOI URL OA? Dimensions (0)
8. Presentation: metajelo, a metadata package for journals to support external linked objects

   Lars Vilhuber and Carl Lagoze

   Feb 2019

   Abs DOI URL OA? Scholar (0) Dimensions (0)

   We propose a metadata package (called metajelo) that is intended to provide academic journals with a lightweight means of registering, at the time of publication, the existence and disposition of supplementary materials. Information about the supplementary materials is, in most cases, critical for the reproducibility and replicability of scholarly results. In many instances, these materials are curated by a third party, which may or may not follow developing standards for the identification and description of those materials. Researchers struggle when attempting to fully comply with data documentation and provenance documentation standards.\textlessbr\textgreater

9. ncrncornell/ced2ar: 2.9.0.0

   Charles Simmer, Ben Perry, Brandon Elam Barker, and 2 more authors

   Feb 2018

   DOI URL Dimensions (0)
10. larsvilhuber/clone-chetty-use-admin-data: Data behind the Chetty (2012) figure on Time Trends in the Use of Administrative Data

    Lars Vilhuber

    Oct 2018

    DOI URL OA? Dimensions (0)
11. Computational Tools for Social Scientists Workshop

    Lars Vilhuber

    2018

    URL
12. Ced²Ar: 2.8.2.0

    Brandon Elam Barker, Charles Simmer, Lars Vilhuber, and 2 more authors

    2017

    Abs DOI Dimensions (0)

    Installation Note: IF you are upgrading an existing installation of CED2AR AND you back up and restore your config files (like we do) THEN you need to add the contents of the patch files here to your existing config files. Instructions are at: Patches/v2.8.2.0 For server installation: ced2ar.war is the server binary BaseX.war is the server database template For desktop installation: ced2ar.jar is the desktop binary ced2ar.sh will run ced2ar.jar The following high level features have been added in this release: New Features and Issues Currently, new features and issues come from two sites: GitHub - The public site where users of the system can post Issues. JIRA - The restricted site used by the development team to track work related to CED2AR development. We try to cross reference a GitHub issue against a jira issue (CDR). New Features The following high level features have been added in this release: Browse by Study - Displays a list of Study Titles (stdyDscr/citation/titlStmt/titl). Clicking on a study title displays the codebook in a tabbed horizontal layout. The tabs are the DDI codebook complex types (Doc, Study, File, Data and Other Material). User Documentation #3 docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl enhancement [CDR-157] - docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl [GitHub issue] UI Navigation Customization - Can set the properties used to display/hide navigation tabs and the names of those tabs. (They are set using the /ced2ar-web/config page.) Administrator Documentation [CDR-157] - docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl [GitHub issue] Global Authentication Option Added - Setting accessMode to AdminOnly allows only users with the ROLE_ADMIN role to access the application. All others are prevented from accessing the pages. This is primarily used for codebook-development servers where crowd-sourced edits are curated and edited by administrators. Administrator Documentation [CDR-189] - Enable global authentication option in web properties Resolved Issues The following issues were fixed in this release. They are listed below. github Issues #26 Possible to delete investigator? #16 Variable edit mode: access levels question #9 Uploads sometimes fail and produce bad error message: invalid XML bug #8 can’t see crowdsourced edits; official version unviewable bug question #6 sha1sum bug #3 docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl enhancement jira Issues Issue [CDR-168] - Improve versioning automation [CDR-180] - ERROR SimpleAsyncUncaughtExceptionHandler...ArrayIndexOutOfBoundsException in ced2ar.eapi.VersionControl [CDR-183] - Configuration page is not updating [CDR-194] - delete investigator [#26] New Feature [CDR-157] - docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl [GitHub issue] [#3] [CDR-189] - Enable global authentication option in web properties QA Fix [CDR-181] - Version control is not commiting or pushing in v2 [CDR-191] - Enable generation of ced2ar.log [CDR-193] - header image for wiki-census

13. Synthetic population housing and person records for the United States

    William Sexton, John M. Abowd, Ian M. Schmutte, and 1 more author

    2017

    DOI OA? Scholar (7) Dimensions (0)
14. Replication Materials for Disclosure Limitation and Confidentality Protection in Linked Data

    Lars Vilhuber, John M. Abowd, and Ian M. Schmutte

    Dec 2017

    itemType: dataset

    DOI URL Scholar (7) Dimensions (0)
15. labordynamicsinstitute/rampnoise: Code for Multiplicative Noise Infusion

    Lars Vilhuber

    Dec 2017

    DOI URL Dimensions (0)
16. Larsvilhuber/Jobcreationblog: Replication For: How Much Do Startups Impact Employment Growth In The U.S.?

    Lars Vilhuber

    2016

    DOI URL OA? Dimensions (0)
17. Usage and outcomes of the Synthetic Data Server

    Lars Vilhuber and John Abowd

    2016

    URL Scholar (8)
18. Replication data for: National estimates of gross employment and job flows from the Quarterly Workforce Indicators with demographic and industry detail

    John M. Abowd and Lars Vilhuber

    2014

    DOI URL OA? Dimensions (0)
19. CED²AR: Comprehensive Extensible Data Documentation and Access Repository

    Benjamin Perry, Jeremy Williams, Lars Vilhuber, and 1 more author

    2013

    tex.howpublished: online resource

    URL
20. NSF-Census Research Network - Cornell node website

    Lars Vilhuber, Benjamin Perry, William Block, and 1 more author

    2012

    URL
21. NSF-Census Research Network

    Alan Karr, Lars Vilhuber, Jamie Nunnelly, and 1 more author

    2012

    URL
22. National Quarterly Workforce Indicators, r2254

    John M. Abowd and Lars Vilhuber

    2012

    URL
23. VirtualRDC - Synthetic Data Server

    John M. Abowd and Lars Vilhuber

    2010

    tex.howpublished: online resource tex.owner: vilhuber tex.timestamp: 2013.10.15

    URL
24. New york state disability and employment status report, 2010

    Sarah Von Schrader, William Erickson, Thomas Golden, and 1 more author

    2010

    URL
25. County-level disability and employment status reports, 2007

    Sarah Von Schrader, William Erickson, Lars Vilhuber, and 1 more author

    2009

    URL
26. County-level disability and employment status reports, 2009

    Sarah Von Schrader, William Erickson, Lars Vilhuber, and 1 more author

    2009

    URL
27. VirtualRDC

    John M. Abowd and Lars Vilhuber

    2004

    tex.howpublished: online resource tex.owner: vilhuber tex.timestamp: 2013.10.15

    URL

Other Publications (0)