Publications

Published Articles

1. A Simulated Reconstruction and Reidentification Attack on the 2010 U.S. Census

   John M. Abowd, Tamara Adams, Robert Ashmead, and 12 more authors

   Harvard Data Science Review, Jul 2025

   DOI URL
2. Using Containers to Validate Research on Confidential Data at Scale

   Lars Vilhuber

   Harvard Data Science Review, Jun 2025

   Abs DOI URL

   I describe past experience with the validation server process over 10 years and several hundred users, as a means to provide proxy access to confidential data. As a modern replacement, I propose the use of containers—simulated computers that encapsulate entire software and file structures. The use of containers ensures reproducibility, reliable portability, and enables scalability. Infrastructure can be outsourced to commercial providers or users, at little to no cost to data providers. The only likely limitation to full automation is the absence of automated output vetting algorithms at statistical agencies.

3. TROV - A Model and Vocabulary for Describing Transparent Research Objects

   Meng Li, Timothy McPhillips, Craig Willis, and 6 more authors

   International Journal of Digital Curation, Feb 2025

   Number: 1

   Abs DOI URL

   The Transparent Research Object Vocabulary (TROV) is a key element of the Transparency Certified (TRACE) approach to ensuring research trustworthiness. In contrast with methods that entail repeating computations in part or in full to verify that the descriptions of methods included in a publication are sufficient to reproduce reported results, the TRACE approach depends on a controlled computing environment termed a Transparent Research System (TRS) to guarantee that accurate, sufficiently complete, and otherwise trustworthy records are captured when results are obtained in the first place. Records identifying (1) the digital artifacts and computations that yielded a research result, (2) the TRS that witnessed the artifacts and supervised the computations, and (3) the specific conditions enforced by the TRS that warrant trust in these records, together constitute a Transparent Research Object (TRO). Digital signatures provided by the TRS and by a trusted third-party timestamp authority (TSA) guarantee the integrity and authenticity of the TRO. The controlled vocabulary TROV provides means to declare and query the properties of a TRO, to enumerate the dimensions of trustworthiness the TRS asserts for a TRO, and to verify that each such assertion is warranted by the documented capabilities of the TRS. Our approach for describing, publishing, and working with TROs imposes no restrictions on how computational artifacts are packaged or otherwise shared, and aims to be interoperable with, rather than to replace, current and future Research Object standards, archival formats, and repository layouts.

4. A Guide for Social Science Journal Editors on Easing into Open Science

   Priya Silverstein, Colin Elman, Amanda Montoya, and 36 more authors

   Research Integrity and Peer Review, Feb 2024

   Abs DOI URL

   Journal editors have a large amount of power to advance open science in their respective fields by incentivizing and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org) has collated several resources on embedding open science in journal editing (www.dpjedi.org/resources). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we have created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx)).

5. Reproduce to validate: A comprehensive study on the reproducibility of economics research

   Sylvérie Herbert, Hautahi Kingi, Flavio Stanchi, and 1 more author

   Canadian Journal of Economics/Revue canadienne d’économique, Aug 2024

   Abs DOI URL

   Abstract Journals have pushed for transparency of research through data availability policies. Such data policies improve availability of data and code, but what is the impact on reproducibility? We present results from a large reproduction exercise for articles published in the American Economic Journal: Applied Economics, which has had a data availability policy since its inception in 2009. Out of 363 published articles, we assessed 274 articles. All articles provided some materials. We excluded 122 articles that required confidential or proprietary data or that required the replicator to otherwise obtain the data (44.5% of assessed articles). We attempted to reproduce 152 articles and were able to fully reproduce the results of 68 (44.7% of attempted reproductions). A further 66 (43.4% of attempted reproductions) were partially reproduced. Many articles required complex code changes even when at least partially reproduced. We collect bibliometric characteristics of authors, but find no evidence for author characteristics as determinants of reproducibility. There does not appear to be a citation bonus for reproducibility. The data availability policy of this journal was effective to ensure availability of materials, but is insufficient to ensure reproduction without additional work by replicators. , Résumé Les journaux militent pour la transparence de la recherche par le biais de politiques sur la disponibilité des données . De telles politiques sur les données améliorent la disponibilité des données et du code, mais quelle en est l’incidence sur la reproductibilité? Nous présentons les résultats d’un grand exercice de reproduction pour des articles publiés dans le American Economic Journal: Applied Economics , qui a une politique de disponibilité des données depuis sa création en 2009. Parmi les 363 articles publiés, nous en évaluons 274. Tous les articles avaient fourni certains documents. Nous avons exclu 122 articles qui nécessitaient des données confidentielles ou exclusives, ou qui exigeaient que la personne chargée de la réplication obtienne autrement les données (44,5% des articles évalués). Nous avons tenté de reproduire 152 articles et avons réussi à entièrement reproduire les résultats de 68 articles (44,7% des reproductions tentées). Nous avons aussi partiellement reproduit 66 autres articles (43,4% des reproductions tentées). De nombreux articles nécessitaient de complexes changements de code, même lorsqu’ils étaient au moins partiellement reproduits. Nous avons recueilli les caractéristiques bibliométriques des auteurs, mais n’avons pas constaté qu’elles étaient déterminantes pour la reproductibilité. La reproductibilité ne semble offrir aucun avantage en matière de citations. La politique de disponibilité des données de ce journal a été efficace pour assurer la disponibilité des documents, mais insuffisante pour assurer la reproduction sans travail supplémentaire des personnes chargées de la réplication.

6. Reinforcing Reproducibility and Replicability: An Introduction

   Lars Vilhuber, Ian Schmutte, Aleksandr Michuda, and 1 more author

   Harvard Data Science Review, Jul 2023

   DOI URL
7. Reproducibility and Transparency versus Privacy and Confidentiality: Reflections from a Data Editor

   Lars Vilhuber

   Journal of Econometrics, Jun 2023

   DOI
8. Introduction to the Special Issue: Models of Linked Employer- Employee Data: Twenty Years after “High Wage Workers and High Wage Firms”

   David Card, Ian Schmutte, and Lars Vilhuber

   Journal of Econometrics, 2023

   Special issue at https://www.sciencedirect.com/journal/journal-of-econometrics/vol/233/issue/2

   DOI URL
9. Teaching for Large-Scale Reproducibility Verification

   Lars Vilhuber, Hyuk Harry Son, Meredith Welch, and 2 more authors

   Journal of Statistics and Data Science Education, Jun 2022

   Abs DOI

   We describe a unique environment in which undergraduate students from various STEM and social science disciplines are trained in data provenance and reproducible methods, and then apply that knowledge to real, conditionally accepted manuscripts and associated replication packages. We describe in detail the recruitment, training, and regular activities. While the activity is not part of a regular curriculum, the skills and knowledge taught through explicit training of reproducible methods and principles, and reinforced through repeated application in a real-life workflow, contribute to the education of these undergraduate students, and prepare them for post-graduation jobs and further studies.

10. An Interview with John M. Abowd

    Ian Schmutte and Lars Vilhuber

    International Statistical Review, Feb 2022

    DOI URL
11. Recalculating - How Uncertainty in Local Labor Market Definitions Affects Empirical Findings

    Andrew Foote, Mark J. Kutzbach, and Lars Vilhuber

    Applied Economics, 2021

    Abs DOI

    This paper evaluates the use of commuting zones as a local labour market definition. We revisit the seminal paper by Tolbert and Sizer and demonstrate the sensitivity of definitions to two features of the methodology: a cluster dissimilarity cut-off, or the count of clusters, and uncertainty in the input data. We show how these features impact empirical estimates using a standard application of commuting zones and an example from related literature. We conclude with advice to researchers on how to demonstrate the robustness of empirical findings to uncertainty in the definition of commuting zones.

12. Report by the AEA Data Editor

    Lars Vilhuber

    AEA Papers and Proceedings, May 2021

    DOI URL
13. Metajelo: A Metadata Package for Journals to Support External Linked Objects

    Lars Vilhuber and Carl Lagoze

    International Journal of Digital Curation, 2021

    DOI
14. Expansion, perspectives, and challenges

    Lars Vilhuber

    Journal of Privacy and Confidentiality, Feb 2021

    DOI URL
15. On Privacy in the Age of COVID-19

    Cynthia Dwork, Alan Karr, Kobbi Nissim, and 1 more author

    Journal of Privacy and Confidentiality, Feb 2021

    DOI URL
16. metajelo: A metadata package for journals to support external linked objects

    Carl Lagoze and Lars Vilhuber

    International Journal of Digital Curation, 2021
17. Total Error and Variability Measures with Integrated Disclosure Limitation for Quarterly Workforce Indicators and LEHD Origin Destination Employment Statistics in OnTheMap

    Kevin L. McKinney, Andrew S. Green, John M. Abowd, and 1 more author

    Journal of Survey Statistics and Methodology, 2020

    Abs DOI

    We report results from the first comprehensive total quality evaluation of five major indicators in the U.S. Census Bureau’s Longitudinal Employer-Household Dynamics (LEHD) Program Quarterly Workforce Indicators (QWI): total employment, beginning-of-quarter employment, full-quarter employment, total payroll, and average monthly earnings of full-quarter employees. Beginning-of-quarter employment is also the main tabulation variable in the LEHD Origin-Destination Employment Statistics (LODES) workplace reports as displayed in OnTheMap (OTM). The evaluation is conducted by generating multiple threads of the edit and imputation models used in the LEHD Infrastructure File System. These threads conform to the Rubin (1987) multiple imputation model, with each thread or implicate being the output of formal probability models that address coverage, edit, and imputation errors. Design-based sampling variability and finite population corrections are also included in the evaluation. We derive special formulas for the Rubin total variability and its components that are consistent with the disclosure avoidance system used for QWI and LODES/OTM workplace reports. These formulas allow us to publish the complete set of detailed total quality measures for QWI and LODES. The analysis reveals that the five publication variables under study are estimated very accurately for tabulations involving at least 10 jobs. Tabulations involving three to nine jobs have quality in the range generally deemed acceptable. Tabulations involving zero, one or two jobs, which are generally suppressed in the QWI and synthesized in LODES, have substantial total variability but their publication in LODES allows the formation of larger custom aggregations, which will in general have the accuracy estimated for tabulations in the QWI based on a similar number of workers.

18. Report by the AEA Data Editor

    Lars Vilhuber, James Turrito, and Keesler Welch

    AEA Papers and Proceedings, May 2020

    DOI URL
19. Applying Data Synthesis for Longitudinal Business Data across Three Countries

    M. Jahangir Alam, Benoit Dostie, Jörg Drechsler, and 1 more author

    Statistics in Transition New Series, 2020

    DOI
20. Reproducibility and Replicability in Economics

    Lars Vilhuber

    Harvard Data Science Review, Dec 2020

    DOI URL
21. Why the Economics Profession Must Actively Participate in the Privacy Protection Debate

    John M. Abowd, Ian M. Schmutte, William N. Sexton, and 1 more author

    AEA Papers and Proceedings, May 2019

    Abs DOI URL

    When Google or the U.S. Census Bureau publish detailed statistics on browsing habits or neighborhood characteristics, some privacy is lost for everybody while supplying public information. To date, economists have not focused on the privacy loss inherent in data publication. In their stead, these issues have been advanced almost exclusively by computer scientists who are primarily interested in technical problems associated with protecting privacy. Economists should join the discussion, first, to determine where to balance privacy protection against data quality; a social choice problem. Furthermore, economists must ensure new privacy models preserve the validity of public data for economic research.

22. Report by the AEA Data Editor

    Lars Vilhuber

    AEA Papers and Proceedings, May 2019

    DOI URL
23. Presentation: metajelo, a metadata package for journals to support external linked objects

    Lars Vilhuber and Carl Lagoze

    Zenodo, Feb 2019

    Abs DOI URL

    We propose a metadata package (called metajelo) that is intended to provide academic journals with a lightweight means of registering, at the time of publication, the existence and disposition of supplementary materials. Information about the supplementary materials is, in most cases, critical for the reproducibility and replicability of scholarly results. In many instances, these materials are curated by a third party, which may or may not follow developing standards for the identification and description of those materials. Researchers struggle when attempting to fully comply with data documentation and provenance documentation standards.\textlessbr\textgreater

24. Effects of a Government-Academic Partnership: Has the NSF-Census Bureau Research Network Helped Improve the U.S. Statistical System?

    Daniel H. Weinberg, John M. Abowd, Robert F. Belli, and 13 more authors

    Journal of Survey Statistics and Methodology, 2018

    Abs DOI

    The National Science Foundation-Census Bureau Research Network (NCRN) was established in 2011 to create interdisciplinary research nodes on methodological questions of interest and significance to the broader research community and to the Federal Statistical System (FSS), particularly the Census Bureau. The activities to date have covered both fundamental and applied statistical research and have focused at least in part on the training of current and future generations of researchers in skills of relevance to surveys and alternative measurement of economic units, households, and persons. This paper discusses some of the key research findings of the eight nodes, organized into six topics: (1) Improving census and survey data collection methods; (2) Using alternative sources of data; (3) Protecting privacy and confidentiality by improving disclosure avoidance; (4) Using spatial and spatio-temporal statistical modeling to improve estimates; (5) Assessing data cost and quality tradeoffs; and (6) Combining information from multiple sources. It also reports on collaborations across nodes and with federal agencies, new software developed, and educational activities and outcomes. The paper concludes with an evaluation of the ability of the FSS to apply the NCRN’s research outcomes and suggests some next steps, as well as the implications of this research-network model for future federal government renewal initiatives.

25. Remembering Stephen Fienberg

    Aleksandra Slavković and Lars Vilhuber

    Journal of Privacy and Confidentiality, Dec 2018

    DOI
26. Relaunching the Journal of Privacy and Confidentiality

    Lars Vilhuber

    Journal of Privacy and Confidentiality, Dec 2018

    DOI
27. Understanding the effect of procedural justice on psychological distress

    Julie Cloutier, Lars Vilhuber, Denis Harrisson, and 1 more author

    International Journal of Stress Management, 2017

    Abs DOI

    Studies on the effect of procedural justice on psychological distress present conflicting results. Drawing on instrumental and relational perspectives of justice, we test the hypothesis that the perception of procedural justice influences the level of workers’ psychological distress. Using a number of validated instruments to collected data from 659 workers in three call centers, we use OLS regressions and Hayes’ PROCESS tool to show that the perception of procedural justice has a direct, unique, and independent effect on psychological distress. The perception of procedural justice has no instrumental role, the key mechanism being the relational role, suggesting that perceived injustice influences psychological distress because it threatens self-esteem. Distributive justice perceptions (recognition, promotions, job security) are not associated with psychological distress, calling into question Siegrist’s model. Our findings suggest that perceived procedural justice provides workers better evidence of the extent to which they are valued and appreciated members of their organizations than do perceptions of distributive justice. The results highlight the greater need for workers to be valued and appreciated for who they are (consideration and esteem), rather than for what they do for their organization (distributive justice of rewards).

28. Making Confidential Data Part of Reproducible Research

    Lars Vilhuber and Carl Lagoze

    Chance, 09/2017 2017

    URL
29. Making confidential data part of reproducible research

    C Lagoze and L Vilhuber

    Chance, 2017

    Abs URL

    The rise of data-centric research practices has uncovered shortcomings in the traditional scholarly communication system. The foundation of that system, the peer-reviewed publication,“[the] selective distribution of ink on paper, or… electronic facsimiles of the same”(Bourne, et al., 2011), does not adequately support what has become an essential element of scholarship; the reproducibility of research results. That is, duplicating a ...

30. Synthetic establishment microdata around the world

    Lars Vilhuber, John M. Abowd, and Jerome P. Reiter

    Statistical Journal of the International Association for Official Statistics, 2016

    Abs DOI

    In contrast to the many public-use microdata samples available for individual and household data from many statistical agencies around the world, there are virtually no establishment or firm microdata available. In large part, this difficulty in providing access to business micro data is due to the skewed and sparse distributions that characterize business data. Synthetic data are simulated data generated from statistical models. We organized sessions at the 2015 World Statistical Congress and the 2015 Joint Statistical Meetings, highlighting work on synthetic establishment microdata. This overview situates those papers, published in this issue, within the broader literature.

31. Using partially synthetic microdata to protect sensitive cells in business statistics

    Javier Miranda and Lars Vilhuber

    Statistical Journal of the International Association for Official Statistics, 2016

    Abs DOI

    We describe and analyze a method that blends records from both observed and synthetic microdata into public-use tabulations on establishment statistics. The resulting tables use synthetic data only in potentially sensitive cells. We describe different algorithms, and present preliminary results when applied to the Census Bureau’s Business Dynamics Statistics and Synthetic Longitudinal Business Database, highlighting accuracy and protection afforded by the method when compared to existing public-use tabulations (with suppressions).

32. Larsvilhuber/Jobcreationblog: Replication For: How Much Do Startups Impact Employment Growth In The U.S.?

    Lars Vilhuber

    2016

    DOI URL
33. A First Step Towards A German SynLBD: Constructing A German Longitudinal Business Database

    Jörg Drechsler and Lars Vilhuber

    Statistical Journal of the IAOS: Journal of the International Association for Official Statistics, 2014

    Abs DOI URL

    One major criticism against the use of synthetic data has been that the efforts necessary to generate useful synthetic data are so in- tense that many statistical agencies cannot afford them. We argue many lessons in this evolving field have been learned in the early years of synthetic data generation, and can be used in the development of new synthetic data products, considerably reducing the required in- vestments. The final goal of the project described in this paper will be to evaluate whether synthetic data algorithms developed in the U.S. to generate a synthetic version of the Longitudinal Business Database (LBD) can easily be transferred to generate a similar data product for other countries. We construct a German data product with infor- mation comparable to the LBD - the German Longitudinal Business Database (GLBD) - that is generated from different administrative sources at the Institute for Employment Research, Germany. In a fu- ture step, the algorithms developed for the synthesis of the LBD will be applied to the GLBD. Extensive evaluations will illustrate whether the algorithms provide useful synthetic data without further adjustment. The ultimate goal of the project is to provide access to multiple synthetic datasets similar to the SynLBD at Cornell to enable comparative studies between countries. The Synthetic GLBD is a first step towards that goal.

34. Looking Back On Three Years Of Using The Synthetic LBD Beta

    Javier Miranda and Lars Vilhuber

    Statistical Journal of the IAOS: Journal of the International Association for Official Statistics, 2014

    Abs DOI URL

    Distributions of business data are typically much more skewed than those for household or individual data and public knowledge of the underlying units is greater. As a results, national statistical offices (NSOs) rarely release establishment or firm-level business microdata due to the risk to respondent confidentiality. One potential approach for overcoming these risks is to release synthetic data where the establishment data are simulated from statistical models designed to mimic the distributions of the real underlying microdata. The US Census Bureau?s Center for Economic Studies in collaboration with Duke University, the National Institute of Statistical Sciences, and Cornell University made available a synthetic public use file for the Longitudinal Business Database (LBD) comprising more than 20 million records for all business establishment with paid employees dating back to 1976. The resulting product, dubbed the SynLBD, was released in 2010 and is the first-ever comprehensive business microdata set publicly released in the United States including data on establishments employment and payroll, birth and death years, and industrial classification. This pa- per documents the scope of projects that have requested and used the SynLBD.

35. Differential Privacy Applications to Bayesian and Linear Mixed Model Estimation

    John M. Abowd, Matthew J. Schneider, and Lars Vilhuber

    Journal of Privacy and Confidentiality, 2013

    Article 4

    Abs URL

    We consider a particular maximum likelihood estimator (MLE) and a computationally intensive Bayesian method for differentially private estimation of the linear mixed-effects model (LMM) with normal random errors. The LMM is important because it is used in small-area estimation and detailed industry tabulations that present significant challenges for confidentiality protection of the underlying data. The differentially private MLE performs well compared to the regular MLE, and deteriorates as the protection increases for a problem in which the small-area variation is at the county level. More dimensions of random effects are needed to adequately represent the time dimension of the data, and for these cases the differentially private MLE cannot be computed. The direct Bayesian approach for the same model uses an informative, reasonably diffuse prior to compute the posterior predictive distribution for the random effects. The empirical differential privacy of this approach is estimated by direct computation of the relevant odds ratios after deleting influential observations according to various criteria.

36. Replicating the Synthetic LBD with German Establishment Data

    Jörg Drechsler and Lars Vilhuber

    Proceedings 59th ISI World Statistics Congress, 25-30 August 2013, Hong Kong (Session STS062) , 2013

    URL
37. Data Management of Confidential Data

    Carl Lagoze, William C. Block, Jeremy Williams, and 2 more authors

    International Journal of Digital Curation, 2013

    Abs DOI

    Social science researchers increasingly make use of data that is confidential because it contains linkages to the identities of people, corporations, etc. The value of this data lies in the ability to join the identifiable entities with external data such as genome data, geospatial information, and the like. However, the confidentiality of this data is a barrier to its utility and curation, making it difficult to fulfill US federal data management mandates and interfering with basic scholarly practices such as validation and reuse of existing results. We describe the complexity of the relationships among data that span a public and private divide. We then describe our work on the CED2AR prototype, a first step in providing researchers with a tool that spans this divide and makes it possible for them to search, access, and cite that data.

38. Data Management of Confidential Data

    Carl Lagoze, William C. Block, Jeremy Williams, and 2 more authors

    International Journal of Digital Curation, 2013
39. Did the Housing Price Bubble Clobber Local Labor Market Job and Worker Flows When It Burst?

    John M. Abowd and Lars Vilhuber

    American Economic Review, May 2012

    Abs DOI URL

    We use the Census Bureau’s Quarterly Workforce Indicators and the Federal Housing Finance Agency’s House Price Indices to study the effects of the housing price bubble on local labor markets. We show that the 35 MSAs in the top decile of the house price boom were most severely impacted. Their stable job employment fell much more than the national average. Their real wage rates did not fall as fast as the national average. Accessions fell much faster than average while separations were constant. Job creations fell substantially while destructions rose slightly.

40. National Estimates of Gross Employment and Job Flows from the Quarterly Workforce Indicators with Demographic and Industry Detail

    John M. Abowd and Lars Vilhuber

    Journal of Econometrics, 2011

    Abs DOI

    The Quarterly Workforce Indicators (QWI) are local labor market data produced and released every quarter by the United States Census Bureau. Unlike any other local labor market series produced in the US or the rest of the world, QWI measure employment flows for workers (accession and separations), jobs (creations and destructions) and earnings for demographic subgroups (age and gender), economic industry (NAICS industry groups), detailed geography (block (experimental), county, Core-Based Statistical Area, and Workforce Investment Area), and ownership (private, all) with fully interacted publication tables. The current QWI data cover 47 states, about 98% of the private workforce in those states, and about 92% of all private employment in the entire economy. State participation is sufficiently extensive to permit us to present the first national estimates constructed from these data. We focus on worker, job, and excess (churning) reallocation rates, rather than on levels of the basic variables. This permits a comparison to existing series from the Job Openings and Labor Turnover Survey and the Business Employment Dynamics Series from the Bureau of Labor Statistics (BLS). The national estimates from the QWI are an important enhancement to existing series because they include demographic and industry detail for both worker and job flow data compiled from underlying micro-data that have been integrated at the job and establishment levels by the Longitudinal Employer-Household Dynamics Program at the Census Bureau. The estimates presented herein were compiled exclusively from public-use data series and are available for download.

41. Science, Confidentiality, and the Public Interest

    John M. Abowd and Lars Vilhuber

    Chance, 2011

    DOI URL
42. Procedural justice criteria in salary determination

    Julie Cloutier and Lars Vilhuber

    Journal of Managerial Psychology, 2008

    Abs DOI URL

    Purpose – The purpose of this research is to identify the dimensionality of the procedural justice construct and the criteria used by employees to assess procedural justice, in the context of salary determination.Design/methodology/approach – Based on a survey of 297 Canadian workers, the paper uses confirmatory factor analysis (CFA) to test the dimensionality and the discriminant and convergent validity of our procedural justice construct. Convergent and predictive validity are also tested using hierarchical linear regressions.Findings – The paper shows the multidimensionality of the procedural justice construct: justice of the salary determination process is assessed through the perceived characteristics of allocation procedures, the perceived characteristics of decision‐makers, and system transparency.Research limitations/implications – Results could be biased towards acceptance; this is discussed. The results also suggest possible extensions to the study.Practical implications – Knowledge of the justice standards improves the ability of organizations to effectively manage the salary determination process and promote its acceptance among employees. Emphasizes the need to adequately manage the selection, training, and perception of decision makers.Originality/value – The paper identifies the standards of procedural justice for salary determination processes. It contributes to the theoretical literature by providing a new multidimensional conceptualization, which helps to better understand the psychological process underlying the perception of procedural justice. The presence of a dimension associated with decision makers is novel and critical for compensation studies.

43. Privacy: Theory meets practice on the map

    Ashwin Machanavajjhala, Daniel Kifer, John M. Abowd, and 2 more authors

    International Conference on Data Engineering (ICDE), 2008

    DOI URL
44. The Sensitivity of Economic Statistics to Coding Errors in Personal Identifiers

    John M. Abowd and Lars Vilhuber

    jbes, Apr 2005

    URL
45. Escaping poverty for low-wage workers: The role of employer characteristics and changes

    Harry Holzer, Julia Lane, and Lars Vilhuber

    Industrial and Labor Relations Review, Jul 2004

    Abs DOI URL

    Using a unique dataset based on individual Unemployment Insurance wage records for Illinois in the 1990s that are matched to other Census data, the authors analyze the extent to which escape from or entry into low earnings among adult workers was associated with changes in their employers and firm characteristics. The results show considerable mobility into and out of low earnings status, even for adults. They indicate that job changes were an important part of the process by which workers escaped or entered low-wage status, and that changes in employer characteristics help to account for these job changes. Matches between personal and firm characteristics also contributed to observed earnings outcomes.

46. Early Career Experiences and Later Career Outcomes: Comparing the United States, France and Germany

    David N. Margolis, Véronique Simonnet, and Lars Vilhuber

    Vierteljahrshefte zur Wirtschaftsforschung, 2001

    Abs DOI URL

    This paper explores the links between individuals’ early career experiences and their labor market outcomes 5 to 20 years later using data from France, (western) Germany, and the United States. Relative to most of the literature, we consider a large set of measures of men’s early career experiences and later career outcomes. Our results differ significantly across countries. Labor market outcomes in Germany are consistent with a dual labor market model. In the case of American workers, either the market learns about unobservable worker characteristics over time or the implicit contracts established at the start of the career are increasingly renegotiated over time. Unobserved heterogeneity in individuals’ networks of labor market contacts is consistent with our results for France. These results reflect optimal firm responses to the different institutional environments in each country in the presence of ex ante imperfect information concerning young workers.

47. La spécificité de la formation en milieu de travail : un survol des contributions théoriques et empiriques récentes,

    Lars Vilhuber

    L’Actualité économique, Revue d’analyse économique, Mar 2001
48. Continuous Training and sectoral mobility in Germany: Evidence from the 90s

    Lars Vilhuber

    Vierteljahresheft für Wirtschaftsforschung, 1999

    Abs URL

    see Vilhuber99b

Working Papers

1. Assessing Utility of Differential Privacy for RCTs

   Kaitlyn R. Webb, Soumya Mukherjee, Aratrika Mustafi, and 2 more authors

   2026

   Version Number: 2

   Abs DOI URL

   Randomized controlled trials (RCTs) have become powerful tools for assessing the impact of interventions and policies in many contexts. They are considered the gold standard for causal inference in the biomedical fields and many social sciences. Researchers have published an increasing number of studies that rely on RCTs for at least part of their inference. These studies typically include the response data that has been collected, de-identified, and sometimes protected through traditional disclosure limitation methods. In this paper, we empirically assess the impact of privacy-preserving synthetic data generation methodologies on published RCT analyses by leveraging available replication packages (research compendia) in economics and policy analysis. We implement three privacy-preserving algorithms, that use as a base one of the basic differentially private (DP) algorithms, the perturbed histogram, to support the quality of statistical inference. We highlight challenges with the straight use of this algorithm and the stability-based histogram in our setting and described the adjustments needed. We provide simulation studies and demonstrate that we can replicate the analysis in a published economics article on privacy-protected data under various parameterizations. We find that relatively straightforward (at a high-level) privacy-preserving methods influenced by DP techniques allow for inference-valid protection of published data. The results have applicability to researchers wishing to share RCT data, especially in the context of low- and middle-income countries, with strong privacy protection.

2. Protecting Confidential Data through Non-Statistical Methods

   Lars Vilhuber

   Oct 2024

   Abs URL

   This chapter will rely on and update previous overviews of how researchers, citizens, and administrators can reliably and securely access confidential data, i.e., data that cannot be simply published as “open data”. I will discuss various legal, technical, and practical ways of securing access to data that is needed for computations. This obviously depends on the type and complexity of the computations but also depends on the who, how, and where access is needed.

3. Mass Reproducibility and Replicability: A New Hope

   Abel Brodeur, Derek Mikola, Nikolai Cook, and 346 more authors

   2024

   Abs URL

   This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness assessments. It reveals several patterns. First, we uncover a high rate of fully computationally reproducible results (over 85%). Second, excluding minor issues like missing packages or broken pathways, we uncover coding errors for about 25% of studies, with some studies containing multiple errors. Third, we test the robustness of the results to 5,511 re-analyses. We find a robustness reproducibility of about 70%. Robustness reproducibility rates are relatively higher for re-analyses that introduce new data and lower for re-analyses that change the sample or the definition of the dependent variable. Fourth, 52% of re-analysis effect size estimates are smaller than the original published estimates and the average statistical significance of a re-analysis is 77% of the original. Lastly, we rely on six teams of researchers working independently to answer eight additional research questions on the determinants of robustness reproducibility. Most teams find a negative relationship between replicators’ experience and reproducibility, while finding no relationship between reproducibility and the provision of intermediate or even raw data combined with the necessary cleaning codes.

4. Reproducibility and Transparency versus Privacy and Confidentiality: Reflections from a Data Editor

   Lars Vilhuber

   2023

   DOI URL
5. The 2010 Census Confidentiality Protections Failed, Here’s How and Why

   John M. Abowd, Tamara Adams, Robert Ashmead, and 11 more authors

   Dec 2023

   arXiv:2312.11283 null

   Abs URL

   Using only 34 published tables, we reconstruct five variables (census block, sex, age, race, and ethnicity) in the confidential 2010 Census person records. Using the 38-bin age variable tabulated at the census block level, at most 20.1% of reconstructed records can differ from their confidential source on even a single value for these five variables. Using only published data, an attacker can verify that all records in 70% of all census blocks (97 million people) are perfectly reconstructed. The tabular publications in Summary File 1 thus have prohibited disclosure risk similar to the unreleased confidential microdata. Reidentification studies confirm that an attacker can, within blocks with perfect reconstruction accuracy, correctly infer the actual census response on race and ethnicity for 3.4 million vulnerable population uniques (persons with nonmodal characteristics) with 95% accuracy, the same precision as the confidential data achieve and far greater than statistical baselines. The flaw in the 2010 Census framework was the assumption that aggregation prevented accurate microdata reconstruction, justifying weaker disclosure limitation methods than were applied to 2010 Census public microdata. The framework used for 2020 Census publications defends against attacks that are based on reconstruction, as we also demonstrate here. Finally, we show that alternatives to the 2020 Census Disclosure Avoidance System with similar accuracy (enhanced swapping) also fail to protect confidentiality, and those that partially defend against reconstruction attacks (incomplete suppression implementations) destroy the primary statutory use case: data for redistricting all legislatures in the country in compliance with the 1965 Voting Rights Act.

6. Teaching for large-scale Reproducibility Verification

   Lars Vilhuber, Hyuk Harry Son, Meredith Welch, and 2 more authors

   Mar 2022

   Abs URL

   We describe a unique environment in which undergraduate students from various STEM and social science disciplines are trained in data provenance and reproducible methods, and then apply that knowledge to real, conditionally accepted manuscripts and associated replication packages. We describe in detail the recruitment, training, and regular activities. While the activity is not part of a regular curriculum, the skills and knowledge taught through explicit training of reproducible methods and principles, and reinforced through repeated application in a real-life workflow, contribute to the education of these undergraduate students, and prepare them for post-graduation jobs and further studies.

7. A template README for social science replication packages

   Lars Vilhuber, Marie Connolly, Miklós Koren, and 2 more authors

   Nov 2022

   Abs URL

   The typical README in social science journals serves the purpose of guiding a reader through the available material and a route to replicating the results in the research paper, including the description of the origins of data and/or description of programs. As such, a good README file should first provide a brief overview of the available material and a brief guide as to how to proceed from beginning to end, before then diving into the specifics. These template files structure such a README in a way that is compliant with the typical data and code workflow in the social sciences.

8. Consumer expectations around COVID-19: Evolution over time

   Fabian Lange and Lars Vilhuber

   2020

   URL
9. Applying Data Synthesis for Longitudinal Business Data across Three Countries

   M. Jahangir Alam, Benoit Dostie, Jörg Drechsler, and 1 more author

   2020

   URL
10. Applying Data Synthesis for Longitudinal Business Data across Three Countries

    M. Jahangir Alam, Benoit Dostie, Jörg Drechsler, and 1 more author

    Jul 2020

    arXiv:2008.02246 [econ]

    Abs DOI URL

    Data on businesses collected by statistical agencies are challenging to protect. Many businesses have unique characteristics, and distributions of employment, sales, and profits are highly skewed. Attackers wishing to conduct identification attacks often have access to much more information than for any individual. As a consequence, most disclosure avoidance mechanisms fail to strike an acceptable balance between usefulness and confidentiality protection. Detailed aggregate statistics by geography or detailed industry classes are rare, public-use microdata on businesses are virtually inexistant, and access to confidential microdata can be burdensome. Synthetic microdata have been proposed as a secure mechanism to publish microdata, as part of a broader discussion of how to provide broader access to such data sets to researchers. In this article, we document an experiment to create analytically valid synthetic data, using the exact same model and methods previously employed for the United States, for data from two different countries: Canada (LEAP) and Germany (BHP). We assess utility and protection, and provide an assessment of the feasibility of extending such an approach in a cost-effective way to other data.

11. Migrating historical AEA supplements

    Lars Vilhuber

    May 2020

    URL
12. Outcomes Report of the Cornell Node of the NSF-Census Research Network

    Lars Vilhuber and William Block

    Jan 2019

    Abs URL

    Description and List of Outcomes of the Cornell node of the NSF-Census Research Network.

13. metajelo: A metadata package for journals to support external linked objects

    Carl Lagoze and Lars Vilhuber

    Apr 2019

    Abs URL

    We propose a metadata package that is intended to provide academic journals with a lightweight means of registering, at the time of publication, the existence and disposition of supplementary materials. Information about the supplementary materials is, in most cases, critical for the reproducibility and replicability of scholarly results. In many instances, these materials are curated by a third party, which may or may not follow developing standards for the identification and description of those materials. As such, the vocabulary described here complements existing initiatives that specify vocabularies to describe the supplementary materials or the repositories and archives in which they have been deposited. Where possible, it reuses elements of relevant other vocabularies, facilitating coexistence with them. Furthermore, it provides an “at publication” record of reproducibility characteristics of a particular article that has been selected for publication. The proposed metadata package documents the key characteristics that journals care about in the case of supplementary materials that are held by third parties: existence, accessibility, and permanence. It does so in a robust, time-invariant fashion at the time of publication, when the editorial decisions are made. It also allows for better documentation of less accessible (non-public data), by treating it symmetrically from the point of view of the journal, therefore increasing the transparency of what up until now has been very opaque.

14. Introductory Readings in Formal Privacy for Economists

    John Abowd, Ian Schmutte, William Sexton, and 1 more author

    Apr 2019

    Supported by Alfred P. Sloan Foundation Grant G-2015-13903 and NSF Grant SES-1131848

    DOI URL
15. Suboptimal Provision of Privacy and Statistical Accuracy When They are Public Goods

    John M. Abowd, Ian M. Schmutte, William Sexton, and 1 more author

    Jun 2019

    Abs DOI URL

    With vast databases at their disposal, private tech companies can compete with public statistical agencies to provide population statistics. However, private companies face different incentives to provide high-quality statistics and to protect the privacy of the people whose data are used. When both privacy protection and statistical accuracy are public goods, private providers tend to produce at least one suboptimally, but it is not clear which. We model a firm that publishes statistics under a guarantee of differential privacy. We prove that provision by the private firm results in inefficiently low data quality in this framework.

16. Cornell Criminal Records Panel Study Questionnaire Wave 2

    Linda Barrington, Esta R. Bigler, Hassan Enayati, and 3 more authors

    Mar 2019

    Abs URL

    Questionnaire used by the Cornell Criminal Records Panel Study. Subjects are members of the class in the law suit against the US Census Bureau. The purpose of this research is to learn more about the errors in background screening and understand how criminal records impact employment opportunities.

17. Introductory Readings in Formal Privacy for Economists

    John Abowd, Ian Schmutte, William Sexton, and 1 more author

    Apr 2019

    DOI URL
18. Why the Economics Profession Must Actively Participate in the Privacy Protection Debate

    John M. Abowd, Ian M. Schmutte, William N. Sexton, and 1 more author

    Jan 2019

    Abs URL

    When Google or the U.S. Census Bureau publish detailed statistics on browsing habits or neighborhood characteristics, some privacy is lost for everybody while supplying public information. To date, economists have not focused on the privacy loss inherent in data publication. In their stead, these issues have been advanced almost exclusively by computer scientists who are primarily interested in technical problems associated with protecting privacy. Economists should join the discussion, first, to determine where to balance privacy protection against data quality; a social choice problem. Furthermore, economists must ensure new privacy models preserve the validity of public data for economic research.

19. Outcomes report \textbar Cornell Node of the NSF-Census Research Network

    Lars Vilhuber and William Block

    Jan 2019

    Abs URL

    Description and List of Outcomes of the Cornell node of the NSF-Census Research Network.

20. Disclosure Limitation and Confidentiality Protection in Linked Data

    John M. Abowd, Ian M. Schmutte, and Lars Vilhuber

    Jan 2018

    Abs DOI URL

    Confidentiality protection for linked administrative data is a combination of access modalities and statistical disclosure limitation. We review traditional statistical disclosure limitation methods and newer methods based on synthetic data, input noise infusion and formal privacy. We discuss how these methods are integrated with access modalities by providing three detailed examples. The first example is the linkages in the Health and Retirement Study to Social Security Administration data. The second example is the linkage of the Survey of Income and Program Participation to administrative data from the Internal Revenue Service and the Social Security Administration. The third example is the Longitudinal Employer-Household Dynamics data, which links state unemployment insurance records for workers and firms to a wide variety of censuses and surveys at the U.S. Census Bureau. For examples, we discuss access modalities, disclosure limitation methods, the effectiveness of those methods, and the resulting analytical validity. The final sections discuss recent advances in access modalities for linked administrative data.

21. LEHD Infrastructure S2014 files in the FSRDC

    Lars Vilhuber

    May 2018

    Abs DOI URL

    The Longitudinal Employer-Household Dynamics (LEHD) Program at the U.S. Census Bureau, with the support of several national research agencies, maintains a set of infrastructure files using administrative data provided by state agencies, enhanced with information from other administrative data sources, demographic and economic (business) surveys and censuses. The LEHD Infrastructure Files provide a detailed and comprehensive picture of workers, employers, and their interaction in the U.S. economy. This document describes the structure and content of the 2014 Snapshot of the LEHD Infrastructure files as they are made available in the Census Bureau’s secure and restricted-access Research Data Center network. The document attempts to provide a comprehensive description of all researcher-accessible files, of their creation, and of any modifications made to the files to facilitate researcher access.

22. Reproducibility and Replicability in Economics

    Lars Vilhuber

    2018

    URL
23. Issues with Unpaywall in Economics

    Lars Vilhuber

    Oct 2018

    URL
24. The Reproducibility of Economics Research: A Case Study

    Hautahi Kingi, Flavio Stanchi, Lars Vilhuber, and 1 more author

    2018

    URL
25. Disclosure Limitation and Confidentiality Protection in Linked Data

    John M. Abowd, Ian M. Schmutte, and Lars Vilhuber

    Jan 2018

    Abs URL

    Confidentiality protection for linked administrative data is a combination of access modalities and statistical disclosure limitation. We review traditional statistical disclosure limitation methods and newer methods based on synthetic data, input noise infusion and formal privacy. We discuss how these methods are integrated with access modalities by providing three detailed examples. The first example is the linkages in the Health and Retirement Study to Social Security Administration data. The second example is the linkage of the Survey of Income and Program Participation to administrative data from the Internal Revenue Service and the Social Security Administration. The third example is the Longitudinal Employer-Household Dynamics data, which links state unemployment insurance records for workers and firms to a wide variety of censuses and surveys at the U.S. Census Bureau. For examples, we discuss access modalities, disclosure limitation methods, the effectiveness of those methods, and the resulting analytical validity. The final sections discuss recent advances in access modalities for linked administrative data.

26. Codebook for the SIPP Synthetic Beta 7.0 (PDF version)

    Lori B. Reeder, Jordan C. Stanley, and Lars Vilhuber

    Nov 2018

    DOI URL
27. Two Perspectives on Commuting: A Comparison of Home to Work Flows Across Job-Linked Survey and Administrative Files

    Andrew S. Green, Mark J. Kutzbach, and Lars Vilhuber

    Jan 2017

    Abs URL

    Commuting flows and workplace employment data have a wide constituency of users including urban and regional planners, social science and transportation researchers, and businesses. The U.S. Census Bureau releases two, national data products that give the magnitude and characteristics of home to work flows. The American Community Survey (ACS) tabulates households’ responses on employment, workplace, and commuting behavior. The Longitudinal Employer-Household Dynamics (LEHD) program tabulates administrative records on jobs in the LEHD Origin-Destination Employment Statistics (LODES). Design differences across the datasets lead to divergence in a comparable statistic: county-to-county aggregate commute flows. To understand differences in the public use data, this study compares ACS and LEHD source files, using identifying information and probabilistic matching to join person and job records. In our assessment, we compare commuting statistics for job frames linked on person, employment status, employer, and workplace and we identify person and job characteristics as well as design features of the data frames that explain aggregate differences. We find a lower rate of within-county commuting and farther commutes in LODES. We attribute these greater distances to differences in workplace reporting and to uncertainty of establishment assignments in LEHD for workers at multi-unit employers. Minor contributing factors include differences in residence location and ACS workplace edits. The results of this analysis and the data infrastructure developed will support further work to understand and enhance commuting statistics in both datasets.

28. Two Perspectives on Commuting: A Comparison of Home to Work Flows Across Job-Linked Survey and Administrative Files

    Andrew S. Green, Mark J. Kutzbach, and Lars Vilhuber

    2017

    Abs URL

    Commuting flows and workplace employment data have a wide constituency of users including urban and regional planners, social science and transportation researchers, and businesses. The U.S. Census Bureau releases two, national data products that give the magnitude and characteristics of home to work flows. The American Community Survey (ACS) tabulates households’ responses on employment, workplace, and commuting behavior. The Longitudinal Employer-Household Dynamics (LEHD) program tabulates administrative records on jobs in the LEHD Origin-Destination Employment Statistics (LODES). Design differences across the datasets lead to divergence in a comparable statistic: county-to-county aggregate commute flows. To understand differences in the public use data, this study compares ACS and LEHD source files, using identifying information and probabilistic matching to join person and job records. In our assessment, we compare commuting statistics for job frames linked on person, employment status, employer, and workplace and we identify person and job characteristics as well as design features of the data frames that explain aggregate differences. We find a lower rate of within-county commuting and farther commutes in LODES. We attribute these greater distances to differences in workplace reporting and to uncertainty of establishment assignments in LEHD for workers at multi-unit employers. Minor contributing factors include differences in residence location and ACS workplace edits. The results of this analysis and the data infrastructure developed will support further work to understand and enhance commuting statistics in both datasets.

29. Two Perspectives on Commuting: A Comparison of Home to Work Flows Across Job-Linked Survey and Administrative Files

    Andrew S. Green, Mark J. Kutzbach, and Lars Vilhuber

    2017

    Abs URL

    Commuting flows and workplace employment data have a wide constituency of users including urban and regional planners, social science and transportation researchers, and businesses. The U.S. Census Bureau releases two, national data products that give the magnitude and characteristics of home to work flows. The American Community Survey (ACS) tabulates households’ responses on employment, workplace, and commuting behavior. The Longitudinal Employer-Household Dynamics (LEHD) program tabulates administrative records on jobs in the LEHD Origin-Destination Employment Statistics (LODES). Design differences across the datasets lead to divergence in a comparable statistic: county-to-county aggregate commute flows. To understand differences in the public use data, this study compares ACS and LEHD source files, using identifying information and probabilistic matching to join person and job records. In our assessment, we compare commuting statistics for job frames linked on person, employment status, employer, and workplace and we identify person and job characteristics as well as design features of the data frames that explain aggregate differences. We find a lower rate of within-county commuting and farther commutes in LODES. We attribute these greater distances to differences in workplace reporting and to uncertainty of establishment assignments in LEHD for workers at multi-unit employers. Minor contributing factors include differences in residence location and ACS workplace edits. The results of this analysis and the data infrastructure developed will support further work to understand and enhance commuting statistics in both datasets.

30. Recalculating - How Uncertainty in Local Labor Market Definitions Affects Empirical Findings

    Andrew Foote, Mark J. Kutzbach, and Lars Vilhuber

    Jan 2017

    Abs DOI URL

    This paper evaluates the use of commuting zones as a local labor market definition. We revisit Tolbert and Sizer (1996) and demonstrate the sensitivity of definitions to two features of the methodology. We show how these features impact empirical estimates using a well-known application of commuting zones. We conclude with advice to researchers using commuting zones on how to demonstrate the robustness of empirical findings to uncertainty in definitions.

31. Recalculating - How Uncertainty in Local Labor Market Definitions Affects Empirical Findings

    Andrew Foote, Mark J. Kutzbach, and Lars Vilhuber

    2017

    Abs URL

    This paper evaluates the use of commuting zones as a local labor market definition. We revisit Tolbert and Sizer (1996) and demonstrate the sensitivity of definitions to two features of the methodology. We show how these features impact empirical estimates using a well-known application of commuting zones. We conclude with advice to researchers using commuting zones on how to demonstrate the robustness of empirical findings to uncertainty in definitions. The analysis, conclusions, and opinions expressed herein are those of the author(s) alone and do not necessarily represent the views of the U.S. Census Bureau or the Federal Deposit Insurance Corporation. All results have been reviewed to ensure that no confidential information is disclosed, and no confidential data was used in this paper. This document is released to inform interested parties of ongoing research and to encourage discussion of work in progress. Much of the work developing this paper occurred while Mark Kutzbach was an employee of the U.S. Census Bureau.

32. Recalculating - How Uncertainty in Local Labor Market Definitions Affects Empirical Findings

    Andrew Foote, Mark J. Kutzbach, and Lars Vilhuber

    2017

    Abs URL

    This paper evaluates the use of commuting zones as a local labor market definition. We revisit Tolbert and Sizer (1996) and demonstrate the sensitivity of definitions to two features of the methodology. We show how these features impact empirical estimates using a well-known application of commuting zones. We conclude with advice to researchers using commuting zones on how to demonstrate the robustness of empirical findings to uncertainty in definitions. The analysis, conclusions, and opinions expressed herein are those of the author(s) alone and do not necessarily represent the views of the U.S. Census Bureau or the Federal Deposit Insurance Corporation. All results have been reviewed to ensure that no confidential information is disclosed, and no confidential data was used in this paper. This document is released to inform interested parties of ongoing research and to encourage discussion of work in progress. Much of the work developing this paper occurred while Mark Kutzbach was an employee of the U.S. Census Bureau.

33. Understanding the effect of procedural justice on psychological distress

    Julie Cloutier, Lars Vilhuber, Denis Harrisson, and 1 more author

    2017

    Abs URL

    Studies on the effect of procedural justice on psychological distress present conflicting results. Drawing on instrumental and relational perspectives of justice, we test the hypothesis that the perception of procedural justice influences the level of workers’ psychological distress. Using a number of validated instruments to collected data from 659 workers in three call centers, we use OLS regressions and Hayes’ PROCESS tool to show that the perception of procedural justice has a direct, unique, and independent effect on psychological distress. The perception of procedural justice has no instrumental role, the key mechanism being the relational role, suggesting that perceived injustice influences psychological distress because it threatens self-esteem. Distributive justice perceptions (recognition, promotions, job security) are not associated with psychological distress, calling into question Siegrist’s model. Our findings suggest that perceived procedural justice provides workers better evidence of the extent to which they are valued and appreciated members of their organizations than do perceptions of distributive justice. The results highlight the greater need for workers to be valued and appreciated for who they are (consideration and esteem), rather than for what they do for their organization (distributive justice of rewards).

34. Total Error and Variability Measures with Integrated Disclosure Limitation for Quarterly Workforce Indicators and LEHD Origin Destination Employment Statistics in On The Map

    Kevin L. McKinney, Andrew S. Green, Lars Vilhuber, and 1 more author

    Jan 2017

    Abs DOI URL

    We report results from the rst comprehensive total quality evaluation of five major indicators in the U.S. Census Bureau’s Longitudinal Employer-Household Dynamics (LEHD) Program Quarterly Workforce Indicators (QWI): total employment, beginning-of-quarter employment, full-quarter employment, total payroll, and average monthly earnings of full-quarter employees. Beginning-of-quarter employment is also the main tabulation variable in the LEHD Origin-Destination Employment Statistics (LODES) workplace reports as displayed in OnTheMap (OTM). The evaluation is conducted by generating multiple threads of the edit and imputation models used in the LEHD Infrastructure File System. These threads conform to the Rubin (1987) multiple imputation model, with each thread or implicate being the output of formal probability models that address coverage, edit, and imputation errors. Design-based sampling variability and nite population corrections are also included in the evaluation. We derive special formulas for the Rubin total variability and its components that are consistent with the disclosure avoidance system used for QWI and LODES/OTM workplace reports. These formulas allow us to publish the complete set of detailed total quality measures for QWI and LODES. The analysis reveals that the five publication variables under study are estimated very accurately for tabulations involving at least 10 jobs. Tabulations involving three to nine jobs have quality in the range generally deemed acceptable. Tabulations involving zero, one or two jobs, which are generally suppressed in the QWI and synthesized in LODES, have substantial total variability but their publication in LODES allows the formation of larger custom aggregations, which will in general have the accuracy estimated for tabulations in the QWI based on a similar number of workers.

35. Effects of a Government-Academic Partnership: Has the NSF-Census Bureau Research Network Helped Improve the U.S. Statistical System?

    Daniel H. Weinberg, John M. Abowd, Robert F. Belli, and 13 more authors

    Jan 2017

    Abs DOI URL

    The National Science Foundation-Census Bureau Research Network (NCRN) was established in 2011 to create interdisciplinary research nodes on methodological questions of interest and significance to the broader research community and to the Federal Statistical System (FSS), particularly the Census Bureau. The activities to date have covered both fundamental and applied statistical research and have focused at least in part on the training of current and future generations of researchers in skills of relevance to surveys and alternative measurement of economic units, households, and persons. This paper discusses some of the key research findings of the eight nodes, organized into six topics: (1) Improving census and survey data collection methods; (2) Using alternative sources of data; (3) Protecting privacy and confidentiality by improving disclosure avoidance; (4) Using spatial and spatio-temporal statistical modeling to improve estimates; (5) Assessing data cost and quality tradeoffs; and (6) Combining information from multiple sources. It also reports on collaborations across nodes and with federal agencies, new software developed, and educational activities and outcomes. The paper concludes with an evaluation of the ability of the FSS to apply the NCRN’s research outcomes and suggests some next steps, as well as the implications of this research-network model for future federal government renewal initiatives.

36. Proceedings from the Synthetic LBD International Seminar

    Lars Vilhuber, Saki Kinney, and Ian Schmutte

    2017

    Abs URL

    On May 9, 2017, we hosted a seminar to discuss the conditions necessary to implement the SynLBD approach with interested parties, with the goal of providing a straightforward toolkit to implement the same procedure on other data. The proceedings summarize the discussions during the workshop. Funding for the workshop was provided by the National Science Foundation (Grants 1012593; 1131848) and the Alfred P. Sloan Foundation (G-2015-13903). Organizational support was provided by the Labor Dynamics Institute at Cornell University.

37. Making Confidential Data Part of Reproducible Research

    Lars Vilhuber and Carl Lagoze

    2017

    URL
38. Proceedings from the 2016 NSF-Sloan Workshop on Practical Privacy

    Lars Vilhuber and Ian Schmutte

    2017

    Abs URL

    Proceedings from the 2016 NSF–Sloan Workshop on Practical Privacy Vilhuber, Lars; Schmutte, Ian; Abowd, John M. On October 14, 2016, we hosted a workshop that brought together economists, survey statisticians, and computer scientists with expertise in the field of privacy preserving methods: Census Bureau staff working on implementing cutting-edge methods in the Bureau’s flagship public-use products mingled with academic researchers from a variety of universities. The four products discussed as part of the workshop were 1. the American Community Survey (ACS); 2. Longitudinal Employer-Household Data (LEHD), in particular the LEHD Origin-Destination Employment Statistics (LODES); the 3. 2020 Decennial Census; and the 4. 2017 Economic Census. The goal of the workshop was to 1. Discuss the specific challenges that have arisen in ongoing efforts to apply formal privacy models to Census data products by drawing together expertise of academic and governmental researchers 2. Produce short written memos that summarize concrete suggestions for practical applications to specific Census Bureau priority areas.

39. Proceedings from the 2017 Cornell-Census-NSF-Sloan Workshop on Practical Privacy

    Lars Vilhuber and Ian Schmutte

    2017

    Abs URL

    These proceedings report on a workshop hosted at the U.S. Census Bureau on May 8, 2017. Our purpose was to gather experts from various backgrounds together to continue discussing the development of formal privacy systems for Census Bureau data products. This workshop was a successor to a previous workshop held in October 2016 (Vilhuber and Schmutte 2017). At our prior workshop, we hosted computer scientists, survey statisticians, and economists, all of whom were experts in data privacy. At that time we discussed the practical implementation of cutting-edge methods for publishing data with formal, provable privacy guarantees, with a focus on applications to Census Bureau data products. The teams developing those applications were just starting out when our first workshop took place, and we spent our time brainstorming solutions to the various problems researchers were encountering, or anticipated encountering. For these cutting-edge formal privacy models, there had been very little effort in the academic literature to apply those methods in real-world settings with large, messy data. We therefore brought together an expanded group of specialists from academia and government who could shed light on technical challenges, subject matter challenges and address how data users might react to changes in data availability and publishing standards. In May 2017, we organized a follow-up workshop, which these proceedings report on. We reviewed progress made in four different areas. The four topics discussed as part of the workshop were 1. the 2020 Decennial Census; 2. the American Community Survey (ACS); 3. the 2017 Economic Census; 4. measuring the demand for privacy and for data quality. As in our earlier workshop, our goals were to 1. Discuss the specific challenges that have arisen in ongoing efforts to apply formal privacy models to Census data products by drawing together expertise of academic and governmental researchers; 2. Produce short written memos that summarize concrete suggestions for practical applications to specific Census Bureau priority areas.

40. Proceedings from the 2016 NSF-Sloan Workshop on Practical Privacy

    Lars Vilhuber and Ian M. Schmutte

    2017

    Abs URL

    On October 14, 2016, we hosted a workshop that brought together economists, survey statisticians, and computer scientists with expertise in the field of privacy preserving methods: Census Bureau staff working on implementing cutting-edge methods in the Bureau’s flagship public-use products mingled with academic researchers from a variety of universities. The four products discussed as part of the workshop were 1. the American Community Survey (ACS); 2. Longitudinal Employer-Household Data (LEHD), in particular the LEHD Origin-Destination Employment Statistics (LODES); the 3. 2020 Decennial Census; and the 4. 2017 Economic Census. The goal of the workshop was to 1. Discuss the specific challenges that have arisen in ongoing efforts to apply formal privacy models to Census data products by drawing together expertise of academic and governmental researchers 2. Produce short written memos that summarize concrete suggestions for practical applications to specific Census Bureau priority areas. Funding for the workshop was provided by the National Science Foundation (CNS-1012593) and the Alfred P. Sloan Foundation. Organizational support was provided by the Research and Methodology Directorate at the U.S. Census Bureau and the Labor Dynamics Institute at Cornell University.

41. Cornell Project for Record Assistance Questionnaire - B-filers

    Linda Barrington, Esta R. Bigler, Hassan Enayati, and 3 more authors

    May 2017

    Abs URL

    Questionnaire used by the Cornell Project for Records Assistance, part of the Cornell University ILR School (Industrial and Labor Relations), to collect information about respondents, their work history, their involvement with the criminal justice system, if any, and their choice of remedy under the settlement of the Gonzalez et al v. Pritzker case. These respondents had chosen not to receive remedy, information was collected to understand their choice.

42. Proceedings from the Synthetic LBD International Seminar

    Lars Vilhuber, Saki Kinney, and Ian Schmutte

    2017

    Abs URL

    On May 9, 2017, we hosted a seminar to discuss the conditions necessary to implement the SynLBD approach with interested parties, with the goal of providing a straightforward toolkit to implement the same procedure on other data. The proceedings summarize the discussions during the workshop.

43. Making Confidential Data Part of Reproducible Research

    Lars Vilhuber and Carl Lagoze

    2017

    URL
44. Utility Cost of Formal Privacy for Releasing National Employer-Employee Statistics

    Samuel Haney, Ashwin Machanavajjhala, John M. Abowd, and 3 more authors

    2017

    Abs URL

    National statistical agencies around the world publish tabular summaries based on combined employer-employee (ER-EE) data. The privacy of both individuals and business establishments that feature in these data are protected by law in most countries. These data are currently released using a variety of statistical disclosure limitation (SDL) techniques that do not reveal the exact characteristics of particular employers and employees, but lack provable privacy guarantees limiting inferential disclosures. In this work, we present novel algorithms for releasing tabular summaries of linked ER-EE data with formal, provable guarantees of privacy. We show that state-of-the-art differentially private algorithms add too much noise for the output to be useful. Instead, we identify the privacy requirements mandated by current interpretations of the relevant laws, and formalize them using the Pufferfish framework. We then develop new privacy definitions that are customized to ER-EE data and satisfy the statutory privacy requirements. We implement the experiments in this paper on production data gathered by the U.S. Census Bureau. An empirical evaluation of utility for these data shows that for reasonable values of the privacy-loss parameter {}epsilon}geq 1, the additive error introduced by our provably private algorithms is comparable, and in some cases better, than the error introduced by existing SDL techniques that have no provable privacy guarantees. For some complex queries currently published, however, our algorithms do not have utility comparable to the existing traditional SDL algorithms. Those queries are fodder for future research.

45. Two Perspectives on Commuting: A Comparison of Home to Work Flows Across Job-Linked Survey and Administrative Files.

    Andrew S. Green, Mark J. Kutzbach, and and Lars Vilhuber

    2017
46. Using Partially Synthetic Microdata to Protect Sensitive Cells in Business Statistics

    Javier Miranda and Lars Vilhuber

    2016

    Abs URL

    We describe and analyze a method that blends records from both observed and synthetic microdata into public-use tabulations on establishment statistics. The resulting tables use synthetic data only in potentially sensitive cells. We describe different algorithms, and present preliminary results when applied to the Census Bureau’s Business Dynamics Statistics and Synthetic Longitudinal Business Database, highlighting accuracy and protection afforded by the method when compared to existing public-use tabulations (with suppressions).

47. Using Partially Synthetic Microdata to Protect Sensitive Cells in Business Statistics

    Javier Miranda and Lars Vilhuber

    Feb 2016

    Abs URL

    We describe and analyze a method that blends records from both observed and synthetic microdata into public-use tabulations on establishment statistics. The resulting tables use synthetic data only in potentially sensitive cells. We describe different algorithms, and present preliminary results when applied to the Census Bureau’s Business Dynamics Statistics and Synthetic Longitudinal Business Database, highlighting accuracy and protection afforded by the method when compared to existing public-use tabulations (with suppressions).

48. Synthetic Establishment Microdata Around the World

    Lars Vilhuber, John A. Abowd, and Jerome P. Reiter

    2016

    Abs URL

    In contrast to the many public-use microdata samples available for individual and household data from many statistical agencies around the world, there are virtually no establishment or firm microdata available. In large part, this difficulty in providing access to business micro data is due to the skewed and sparse distributions that characterize business data. Synthetic data are simulated data generated from statistical models. We organized sessions at the 2015 World Statistical Congress and the 2015 Joint Statistical Meetings, highlighting work on synthetic establishment microdata. This overview situates those papers, published in this issue, within the broader literature.

49. A First Step Towards A German SynLBD: Constructing A German Longitudinal Business Database

    Jörg Drechsler and Lars Vilhuber

    Feb 2014

    Abs URL

    One major criticism against the use of synthetic data has been that the efforts necessary to generate useful synthetic data are so in- tense that many statistical agencies cannot afford them. We argue many lessons in this evolving field have been learned in the early years of synthetic data generation, and can be used in the development of new synthetic data products, considerably reducing the required in- vestments. The final goal of the project described in this paper will be to evaluate whether synthetic data algorithms developed in the U.S. to generate a synthetic version of the Longitudinal Business Database (LBD) can easily be transferred to generate a similar data product for other countries. We construct a German data product with infor- mation comparable to the LBD - the German Longitudinal Business Database (GLBD) - that is generated from different administrative sources at the Institute for Employment Research, Germany. In a fu- ture step, the algorithms developed for the synthesis of the LBD will be applied to the GLBD. Extensive evaluations will illustrate whether the algorithms provide useful synthetic data without further adjustment. The ultimate goal of the project is to provide access to multiple synthetic datasets similar to the SynLBD at Cornell to enable comparative studies between countries. The Synthetic GLBD is a first step towards that goal.

50. Looking Back On Three Years Of Using The Synthetic LBD Beta

    Javier Miranda and Lars Vilhuber

    Feb 2014

    Abs URL

    Distributions of business data are typically much more skewed than those for household or individual data and public knowledge of the underlying units is greater. As a results, national statistical offices (NSOs) rarely release establishment or firm-level business microdata due to the risk to respondent confidentiality. One potential approach for overcoming these risks is to release synthetic data where the establishment data are simulated from statistical models designed to mimic the distributions of the real underlying microdata. The US Census Bureau?s Center for Economic Studies in collaboration with Duke University, the National Institute of Statistical Sciences, and Cornell University made available a synthetic public use file for the Longitudinal Business Database (LBD) comprising more than 20 million records for all business establishment with paid employees dating back to 1976. The resulting product, dubbed the SynLBD, was released in 2010 and is the first-ever comprehensive business microdata set publicly released in the United States including data on establishments employment and payroll, birth and death years, and industrial classification. This pa- per documents the scope of projects that have requested and used the SynLBD.

51. LEHD Infrastructure files in the Census RDC - Overview

    Lars Vilhuber and Kevin McKinney

    Jun 2014

    Abs URL

    The Longitudinal Employer-Household Dynamics (LEHD) Program at the U.S. Census Bureau, with the support of several national research agencies, maintains a set of infrastructure files using administrative data provided by state agencies, enhanced with information from other administrative data sources, demographic and economic (business) surveys and censuses. The LEHD Infrastructure Files provide a detailed and comprehensive picture of workers, employers, and their interaction in the U.S. economy. This document describes the structure and content of the 2011 Snapshot of the LEHD Infrastructure files as they are made available in the Census Bureaus secure and restricted-access Research Data Center network. The document attempts to provide a comprehensive description of all researcher-accessible files, of their creation, and of any modifcations made to the files to facilitate researcher access.

52. LEHD Infrastructure files in the Census RDC - Overview

    Lars Vilhuber and Kevin McKinney

    Jun 2014

    Abs URL

    The Longitudinal Employer-Household Dynamics (LEHD) Program at the U.S. Census Bureau, with the support of several national research agencies, maintains a set of infrastructure files using administrative data provided by state agencies, enhanced with information from other administrative data sources, demographic and economic (business) surveys and censuses. The LEHD Infrastructure Files provide a detailed and comprehensive picture of workers, employers, and their interaction in the U.S. economy. This document describes the structure and content of the 2011 Snapshot of the LEHD Infrastructure files as they are made available in the Census Bureaus secure and restricted-access Research Data Center network. The document attempts to provide a comprehensive description of all researcher-accessible files, of their creation, and of any modifcations made to the files to facilitate researcher access.

53. Estimation de la contribution de la réallocation de la main-d’oeuvre à la croissance de la productivité au Canada

    Charles Bérubé, Benoit Dostie, and Lars Vilhuber

    2013

    Abs URL

    In this report, we estimate the contribution of labour reallocation to productivity growth in the Canadian manufacturing sector. We find that most of productivity growth comes from within firm improvements, leaving a limited role for labour reallocation. Still, we also find that the importance of labour reallocation increase over time. This is both due to increasing net-entry and inter-firm effects. These effects are much more important post 2000 than in the 1990s. We also find that lost production from exiting firms is now most likely replaced by production from existing firms, while previously, it was more likely to be replaced by production from new firms. (French only)

54. Methods for Protecting the Confidentiality of Firm-Level Data: Issues and Solutions

    Lars Vilhuber

    Mar 2013

    Abs URL

    This report will provide an overview of methods used by statistical agencies to encourage, support, and enhance research access to data for the purpose of generating new knowledge. Quite a few reports and scientific articles have addressed the issue before, and we will be highly indebted to that literature. To a summary of that literature, we hope to provide some recent developments and experiences derived from a decade of working with systems that increase access as both researchers as well as data providers. The report will focus on the data provided by statistical agencies, but it should be understood that government agencies other than a National Statistical Office (NSO) may acquire that function. While excluding the legal background limiting or permitting such data collection and provision, we will highlight some alternate sources and methods, prior to concluding.

55. Methods for Protecting the Confidentiality of Firm-Level Data: Issues and Solutions

    Lars Vilhuber

    Mar 2013

    Abs URL

    This report will provide an overview of methods used by statistical agencies to encourage, support, and enhance research access to data for the purpose of generating new knowledge. Quite a few reports and scientific articles have addressed the issue before, and we will be highly indebted to that literature. To a summary of that literature, we hope to provide some recent developments and experiences derived from a decade of working with systems that increase access as both researchers as well as data providers. The report will focus on the data provided by statistical agencies, but it should be understood that government agencies other than a National Statistical Office (NSO) may acquire that function. While excluding the legal background limiting or permitting such data collection and provision, we will highlight some alternate sources and methods, prior to concluding.

56. Dynamically Consistent Noise Infusion and Partially Synthetic Data as Confidentiality Protection Measures for Related Time Series

    John M. Abowd, Kaj Gittings, Kevin L. McKinney, and 3 more authors

    2012

    URL
57. National Quarterly Workforce Indicators, r2254

    John M. Abowd and Lars Vilhuber

    2012

    URL
58. Dynamically Consistent Noise Infusion and Partially Synthetic Data as Confidentiality Protection Measures for Related Time Series

    John M Abowd, R Kaj Kaj Gittings, Kevin L McKinney, and 3 more authors

    Oct 2012

    Abs DOI URL

    The Census Bureau’s Quarterly Workforce Indicators (QWI) provide detailed quarterly statistics on employment measures such as worker and job flows, tabulated by

59. LEHD Infrastructure Files in the Census RDC: Overview of S2004 Snapshot

    Kevin McKinney and Lars Vilhuber

    Apr 2011

    Abs URL

    The Longitudinal Employer-Household Dynamics (LEHD) Program at the U.S. Census Bureau, with the support of several national research agencies, has built a set of infrastructure files using administrative data provided by state agencies, enhanced with information from other administrative data sources, demographic and economic (business) surveys and censuses. The LEHD Infrastructure Files provide a detailed and comprehensive picture of workers, employers, and their interaction in the U.S. economy. This document describes the structure and content of the 2004 Snapshot of the LEHD Infrastructure files as they are made available in the Census Bureau’s Research Data Center network.

60. LEHD Data Documentation LEHD-OVERVIEW-S2008-rev1

    Kevin McKinney and Lars Vilhuber

    Dec 2011

    Abs URL

    No abstract is available for this item.

61. New York State Disability and Employment Status Report, 2011

    Sarah Von Schrader, William Erickson, Thomas Golden, and 1 more author

    2011

    URL
62. LEHD Infrastructure files in the Census RDC-Overview

    Kevin L. McKinney and Lars Vilhuber

    2011

    URL
63. National Estimates of Gross Employment and Job Flows from the Quarterly Workforce Indicators with Demographic and Industry Detail (with color graphs)

    John M. Abowd and Lars Vilhuber

    Jun 2010

    Abs URL

    The Quarterly Workforce Indicators (QWI) are local labor market data produced and released every quarter by the United States Census Bureau. Unlike any other local labor market series produced in the U.S. or the rest of the world, the QWI measure employment flows for workers (accession and separations), jobs (creations and destructions) and earnings for demographic subgroups (age and gender), economic industry (NAICS industry groups), detailed geography (block (experimental), county, Core- Based Statistical Area, and Workforce Investment Area), and ownership (private, all) with fully interacted publication tables. The current QWI data cover 47 states, about 98% of the private workforce in those states, and about 92% of all private employment in the entire economy. State participation is sufficiently extensive to permit us to present the first national estimates constructed from these data. We focus on worker, job, and excess (churning) reallocation rates, rather than on levels of the basic variables. This permits comparison to existing series from the Job Openings and Labor Turnover Survey and the Business Employment Dynamics Series from the Bureau of Labor Statistics. The national estimates from the QWI are an important enhancement to existing series because they include demographic and industry detail for both worker and job flow data compiled from underlying micro-data that have been integrated at the job and establishment levels by the Longitudinal Employer-Household Dynamics Program at the Census Bureau. The estimates presented herein were compiled exclusively from public-use data series and are available for download.

64. Measuring firm-level displacement events with administrative data

    Lars Vilhuber

    2010
65. New York State Disability and Employment Status Report, 2009

    Sarah Von Schrader, William Erickson, Lars Vilhuber, and 1 more author

    2010

    URL
66. Adjusting imperfect data: Overview and case studies

    Lars Vilhuber

    2007

    DOI URL
67. Confidentiality Protection in the Census Bureau’s Quarterly Workforce Indicators

    John M. Abowd, Bryce E. Stephens, and Lars Vilhuber

    2005
68. Adjusting imperfect data: Overview and case studies

    Lars Vilhuber

    2004
69. The Creation of the Employment Dynamics Estimates

    John M. Abowd, Paul A. Lengermann, and Lars Vilhuber

    2002

    URL
70. The Sensitivity of Economic Statistics to Coding Errors in Personal Identifiers

    John M. Abowd and Lars Vilhuber

    2002
71. Displaced workers, early leavers, and re-employment wages

    Audra Bowlus and Lars Vilhuber

    2002
72. Abandoning the Sinking Ship: The Composition of Worker Flows Prior to Displacement

    Paul A. Lengermann and Lars Vilhuber

    2002
73. Escaping poverty for low-wage workers: The role of employer characteristics and changes

    Harry J. Holzer, Julia I. Lane, Lars Vilhuber, and 2 more authors

    2001
74. Longitudinal analysis of SSN response on SIPP 1990-1993 panels

    Lars Vilhuber and Robert Pedace

    2000
75. Continuous Training and sectoral mobility in Germany

    Lars Vilhuber

    1999

    Abs

    This article studies mobility patterns of German workers in light of a model of sector-specific human capital. Furthermore, I employ and describe little-used data on continuous on-the-job training occuring after apprenticeships. Results are presented describing the incidence and duration of continuous training. Continuous training is quite common, depite the high incidence of apprenticeships which precedes this part of a worker’s career. Most previous studies have only distinguished between firm-specific and general human capital, generally concluding that training was general. Inconsistent with those conclusions, I show that German men are more likely to find a job within the same sector if they have received continuous training in that sector. These results are similar to results obtained for young U.S. workers, and suggest that sector-specific capital is an important feature of very different labor markets. Furthermore, the results suggest that the observed effect of training on mobility is sensitive to the state of the business cycle, indicating a more complex interaction between supply and demand that most theoretical models allow for.

76. Sector-Specific On-The-Job Training: Evidence from U.S. Data

    Lars Vilhuber

    1997

    Abs

    Using data from the National Longitudinal Survey of Youth (NLSY), we re-examine the effect of formal on-the-job training on mobility patterns of young American workers. By employing parametric duration models, we evaluate the economic impact of training on productive time with an employer. Confirming previous studies, we find a positive and statistically significant impact of formal on-the-job training on tenure with the employer providing the training. However, expected duration net of the time spent in the training program is generally not significantly increased. We proceed to document and analyze intra-sectoral and cross-sectoral mobility patterns in order to infer whether training provides firm-specific, industry-specific, or general human capital. The econometric analysis rejects a sequential model of job separation in favor of a competing risks specification. We find significant evidence for the industry-specificity of training. The probability of sectoral mobility upon job separation decreases with training received in the current industry, whether with the last employer or previous employers, and employment attachment increases with on-the-job training. These results are robust to a number of variations on the base model.

77. Wage Flexibility and Contract Structure in Germany

    Lars Vilhuber

    1996

    Abs

    In this paper, we look at how labor market conditions at different points during the tenure of individuals with firms are correlated with current earnings. Using data from the German Socioeconomic Panel on individuals for the period 1984 to 1994, we find that both the contemporaneous unemployment rate and prior values of the unemployment rate are significantly correlated with current earnings, contrary to results for the American labor market. We interpret this result as evidence that German unions do in fact bargain over both wages and employment, but that the models of individualistic contracts, such as the implicit contract model, may explain some of the observed wage drift and longer-term wage movements reasonably well. Furthermore, we explore the heterogeneity of contracts over a variety of worker and job characteristics. In particular, we find evidence that contracts differ across industries and across firm size. Workers of large firms are remarkably more insulated from the job market than workers for any other type of firm, indicating the importance of internal job markets.

Conference Papers

1. Report of the AEA Data Editor

   Lars Vilhuber and Jack Cavanagh

   In AEA Papers and Proceedings, May 2025

   tex.ids= vilhuber2025

   DOI URL
2. Report of the AEA Data Editor

   Lars Vilhuber

   In AEA Papers and Proceedings, May 2024

   DOI URL
3. Report of the AEA Data Editor

   Lars Vilhuber

   In AEA Papers and Proceedings, May 2023

   DOI URL
4. Report by the AEA Data Editor

   Lars Vilhuber

   In AEA Papers and Proceedings, May 2022

   DOI
5. Making Confidential Data Part of Reproducible Research

   Lars Vilhuber

   In Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop, 2019

   National Academies of Sciences, Engineering, and Medicine; Michael Cohen (Rapporteur)

   Abs DOI URL

   In 2014 the National Science Foundation (NSF) provided support to the National Academies of Sciences, Engineering, and Medicine for a series of Forums on Open Science in response to a government-wide directive to support increased public access to the results of research funded by the federal government. However, the breadth of the work resulting from the series precluded a focus on any specific topic or discussion about how to improve public access. Thus, the main goal of the Workshop on Transparency and Reproducibility in Federal Statistics was to develop some understanding of what principles and practices are, or would be, supportive of making federal statistics more understandable and reviewable, both by agency staff and the public. This publication summarizes the presentations and discussions from the workshop.

6. Utility Cost of Formal Privacy for Releasing National Employer-Employee Statistics

   Samuel Haney, Ashwin Machanavajjhala, John M. Abowd, and 3 more authors

   In Proceedings of the 2017 International Conference on Management of Data, 2017

   Abs DOI URL

   National statistical agencies around the world publish tabular summaries based on combined employer-employee (ER-EE) data. The privacy of both individuals and business establishments that feature in these data are protected by law in most countries. These data are currently released using a variety of statistical disclosure limitation (SDL) techniques that do not reveal the exact characteristics of particular employers and employees, but lack provable privacy guarantees limiting inferential disclosures. In this work, we present novel algorithms for releasing tabular summaries of linked ER-EE data with formal, provable guarantees of privacy. We show that state-of-the-art differentially private algorithms add too much noise for the output to be useful. Instead, we identify the privacy requirements mandated by current interpretations of the relevant laws, and formalize them using the Pufferfish framework. We then develop new privacy definitions that are customized to ER-EE data and satisfy the statutory privacy requirements. We implement the experiments in this paper on production data gathered by the U.S. Census Bureau. An empirical evaluation of utility for these data shows that for reasonable values of the privacy-loss parameter ε≥1, the additive error introduced by our provably private algorithms is comparable, and in some cases better, than the error introduced by existing SDL techniques that have no provable privacy guarantees. For some complex queries currently published, however, our algorithms do not have utility comparable to the existing traditional SDL algorithms. Those queries are fodder for future research.

7. CED2AR: The Comprehensive Extensible Data Documentation and Access Repository

   Carl Lagoze, Lars Vilhuber, Jeremy Williams, and 2 more authors

   In ACM/IEEE Joint Conference on Digital Libraries (JCDL 2014), 8th-12th september 2014 2014

   Presented at the ACM/IEEE Joint Conference on Digital Libraries (JCDL 2014)

   Abs

   Social science researchers increasingly make use of data that is confidential because it contains linkages to the identities of people, corporations, etc. The value of this data lies in the ability to join the identifiable entities with external data such as genome data, geospatial information, and the like. However, the confidentiality of this data is a barrier to its utility and curation, making it difficult to fulfill US federal data management mandates and interfering with basic scholarly practices such as validation and reuse of existing results. We describe the complexity of the relationships among data that span a public and private divide. We then describe our work on the CED2AR prototype, a first step in providing researchers with a tool that spans this divide and makes it possible for them to search, access, and cite that data.

8. CED2AR: The Comprehensive Extensible Data Documentation and Access Repository.

   Carl Lagoze, Lars Vilhuber, Jeremy Williams, and 2 more authors

   In ACM/IEEE Joint Conference on Digital Libraries (JCDL 2014), 2014

   Abs DOI URL

   We describe the design, implementation, and deployment of the Comprehensive Extensible Data Documentation and Access Repository (CED2AR). This is a metadata repository system that allows researchers to search, browse, access, and cite confidential data and metadata through either a web-based user interface or programmatically through a search API, all the while re-reusing and linking to existing archive and provider generated metadata. CED2AR is distinguished from other metadata repository-based applications due to requirements that derive from its social science context. These include the need to cloak confidential data and metadata and manage complex provenance chains.

9. Encoding Provenance of Social Science Data: Integrating PROV with DDI

   Carl Lagoze, William C. Block, Jeremy Williams, and 1 more author

   In 5th Annual European DDI User Conference, 2013

   Abs

   Provenance is a key component of evaluating the integrity and reusability of data for scholarship. While recording and providing access provenance has always been important, it is even more critical in the web environment in which data from distributed sources and of varying integrity can be combined and derived. The PROV model, developed under the auspices of the W3C, is a foundation for semantically-rich, interoperable, and web-compatible provenance metadata. We report on the results of our experimentation with integrating the PROV model into the DDI metadata for a complex, but characteristic, example social science data. We also present some preliminary thinking on how to visualize those graphs in the user interface.

10. Encoding Provenance Metadata for Social Science Datasets

    Carl Lagoze, Jeremy Willliams, and Lars Vilhuber

    In Metadata and Semantics Research, 2013

    Abs DOI URL

    Recording provenance is a key requirement for data-centric scholarship, allowing researchers to evaluate the integrity of source data sets and reproduce, and thereby, validate results. Provenance has become even more critical in the web environment in which data from distributed sources and of varying integrity can be combined and derived. Recent work by the W3C on the PROV model provides the foundation for semantically-rich, interoperable, and web-compatible provenance metadata. We apply that model to complex, but characteristic, provenance examples of social science data, describe scenarios that make scholarly use of those provenance descriptions, and propose a manner for encoding this provenance metadata within the widely-used DDI metadata standard.

11. A Proposed Solution to the Archiving and Curation of Confidential Scientific Inputs

    J.M. Abowd, Lars Vilhuber, and William C. Block

    In Privacy in Statistical Databases, 2012

    URL

Book Chapters

1. Protecting Confidential Data through Non-Statistical Methods

   Lars Vilhuber

   In Handbook of sharing confidential data: differential privacy, secure multiparty computation, and synthetic data, 2024

   Abs

   "Statistical agencies, research organizations, companies, and other data stewards that seek to share data with the public face a challenging dilemma. They need to protect the privacy and confidentiality of data subjects’ and their attributes while providing data products that are useful for their intended purposes. In an age when information on data subjects is available from a wide range of data sources, as are the computational resources to obtain that information, this challenge is increasingly difficult. The Handbook of Sharing Confidential Data helps data stewards understand how tools from the data confidentiality literature-specifically, synthetic data, formal privacy, and secure computation-can be used to manage trade-offs in disclosure risk and data usefulness"–

2. Disclosure Limitation and Confidentiality Protection in Linked Data

   John M. Abowd, Ian M. Schmutte, and Lars Vilhuber

   In Administrative Records for Survey Methodology, 2021

   Abs DOI

   Confidentiality protection for linked administrative data is a combination of access modalities and statistical disclosure limitation. We review traditional statistical disclosure limitation methods and newer methods based on synthetic data, input noise infusion and formal privacy. We discuss how these methods are integrated with access modalities by providing three detailed examples. The first example is the linkages in the Health and Retirement Study to Social Security Administration data. The second example is the linkage of the Survey of Income and Program Participation to administrative data from the Internal Revenue Service and the Social Security Administration. The third example is the Longitudinal Employer-Household Dynamics data, which links state unemployment insurance records for workers and firms to a wide variety of censuses and surveys at the U.S. Census Bureau. For examples, we discuss access modalities, disclosure limitation methods, the effectiveness of those methods, and the resulting analytical validity. The final sections discuss recent advances in access modalities for linked administrative data.

3. Synthetic Data via Quantile Regression for Heavy-Tailed and Heteroskedastic Data

   Michelle Pistner, Aleksandra Slavković, and Lars Vilhuber

   In Privacy in Statistical Databases, 2018

   DOI URL
4. Synthetic Longitudinal Business Databases for International Comparisons

   Jörg Drechsler and Lars Vilhuber

   In Privacy in Statistical Databases, 2014

   Abs DOI URL

   International comparison studies on economic activity are often hampered by the fact that access to business microdata is very limited on an international level. A recently launched project tries to overcome these limitations by improving access to Business Censuses from multiple countries based on synthetic data. Starting from the synthetic version of the longitudinally edited version of the U.S. Business Register (the Longitudinal Business Database, LBD), the idea is to create similar data products in other countries by applying the synthesis methodology developed for the LBD to generate synthetic replicates that could be distributed without confidentiality concerns. In this paper we present some first results of this project based on German business data collected at the Institute for Employment Research.

5. Using Partially Synthetic Data to Replace Suppression in the Business Dynamics Statistics: Early Results

   Javier Miranda and Lars Vilhuber

   In Privacy in Statistical Databases, 2014

   Abs DOI URL

   The Business Dynamics Statistics is a product of the U.S. Census Bureau that provides measures of business openings and closings, and job creation and destruction, by a variety of cross-classifications (firm and establishment age and size, industrial sector, and geography). Sensitive data are currently protected through suppression. However, as additional tabulations are being developed, at ever more detailed geographic levels, the number of suppressions increases dramatically. This paper explores the option of providing public-use data that are analytically valid and without suppressions, by leveraging synthetic data to replace observations in sensitive cells.

6. Encoding Provenance Metadata for Social Science Datasets.

   Carl Lagoze, Jeremy Willliams, and and Lars Vilhuber

   In Metadata and Semantics Research. Communications in Computer and Information Science 390. Springer International Publishing, 2013
7. A Proposed Solution to the Archiving and Curation of Confidential Scientific Inputs

   John M. Abowd, Lars Vilhuber, and William Block

   In Privacy in Statistical Databases, 2012

   DOI URL
8. The LEHD Infrastructure Files and the Creation of the Quarterly Workforce Indicators

   J M Abowd, B E Stephens, L Vilhuber, and 4 more authors

   In Producer Dynamics: New Evidence from Micro Data, 2009
9. How Protective Are Synthetic Data

   John Abowd and Lars Vilhuber

   In Privacy in Statistical Database, Sep 2008

   DOI URL
10. How Did Universal Primary Education Affect Returns to Education and Labor Market Participation in Uganda?

    Lisa Dragoset and Lars Vilhuber

    2008

    DOI

Books

1. Synthetic population housing and person records for the United States

   William Sexton, John M. Abowd, Ian M. Schmutte, and 1 more author

   2017

   DOI
2. labordynamicsinstitute/rampnoise: Code for Multiplicative Noise Infusion

   Lars Vilhuber

   Dec 2017

   DOI URL

Data & Software

1. Assessing Reproducibility in Economics Using Standardized Crowd-sourced Analysis

   Abel Brodeur, Seung Yong Sung, Edward Miguel, and 2 more authors

   May 2025

   Abs DOI URL

   This paper presents a framework to standardize crowd-sourced computational reproductions in economics through the Social Science Reproduction Platform (SSRP). The approach address four main challenges for computational reproductions: a lack of standardization, aggregation issues, existing incentives for “adversarial” interactions, and the loss of knowledge from analyses that are never published. We then summarize the first 487 reproductions uploaded on the SSRP. The results show substantial heterogeneity in the ability to successfully reproduce empirical results in economics research, with approximately 30% of recent studies meeting at least a basic definition of being computationally reproducible.

2. Crowdsourcing Digital Public Goods: A Field Experiment on Metadata Contributions

   Linfeng Li, Yan Chen, Margaret C. Levenstein, and 1 more author

   Nov 2024

   Abs DOI URL

   This study explores why people choose to contribute metadata, which is data about data. Using a field experiment conducted with more than 3,000 authors of AEA journal articles, our control message reduces the uncertainty about the future value of metadata, whereas those from the treatment conditions additionally make the private or social benefits of metadata salient. Surprisingly, we find that participants in the control condition provide significantly more metadata compared to those in the treatments. This suggests that simply knowing that metadata will have value is sufficient to motivate people to contribute. Our results also highlight the importance of interface design in online field experiments.

3. Uncertainty in times of COVID-19: Raw survey data

   Fabian Lange, Lars Vilhuber, and Nicholas Gordon

   Jul 2020

   DOI URL
4. Replication code and data for: Recalculating ... How Uncertainty in Local Labor Market Definitions Affects Empirical Findings

   Andrew D. Foote, Mark J. Kutzbach, and Lars Vilhuber

   Oct 2020

   See also https://larsvilhuber.github.io/MobZ/README.html.

   DOI URL
5. Applying Data Synthesis for Longitudinal Business Data across Three Countries

   M. Jahangir Alam, Benoit Dostie, Jörg Drechsler, and 1 more author

   May 2020

   See also https://labordynamicsinstitute.github.io/SyntheticLEAP/.

   DOI URL
6. Data for: Requesting replication materials via email

   Lars Vilhuber

   Nov 2020

   Abs DOI URL

   No description provided.

7. labordynamicsinstitute/readin_qcew_sas: A sequence of programs to readin in QCEW data from the Bureau of Labor Statistics

   Lars Vilhuber and Melissa Bjelland

   Jun 2020

   Abs DOI URL

   These programs download and readin bulk data from the U.S. Bureau of Labor Statistics for the Quarterly Census of Employment and Wages data in ENB format.

8. ncrncornell/ced2ar: 2.9.0.0

   Charles Simmer, Ben Perry, Brandon Elam Barker, and 2 more authors

   Feb 2018

   DOI URL
9. larsvilhuber/clone-chetty-use-admin-data: Data behind the Chetty (2012) figure on Time Trends in the Use of Administrative Data

   Lars Vilhuber

   Oct 2018

   DOI URL
10. Computational Tools for Social Scientists Workshop

    Lars Vilhuber

    2018

    URL
11. Ced²Ar: 2.8.2.0

    Brandon Elam Barker, Charles Simmer, Lars Vilhuber, and 2 more authors

    2017

    tex.publisher= Zenodo

    Abs DOI URL

    Installation Note: IF you are upgrading an existing installation of CED2AR AND you back up and restore your config files (like we do) THEN you need to add the contents of the patch files here to your existing config files. Instructions are at: Patches/v2.8.2.0 For server installation: ced2ar.war is the server binary BaseX.war is the server database template For desktop installation: ced2ar.jar is the desktop binary ced2ar.sh will run ced2ar.jar The following high level features have been added in this release: New Features and Issues Currently, new features and issues come from two sites: GitHub - The public site where users of the system can post Issues. JIRA - The restricted site used by the development team to track work related to CED2AR development. We try to cross reference a GitHub issue against a jira issue (CDR). New Features The following high level features have been added in this release: Browse by Study - Displays a list of Study Titles (stdyDscr/citation/titlStmt/titl). Clicking on a study title displays the codebook in a tabbed horizontal layout. The tabs are the DDI codebook complex types (Doc, Study, File, Data and Other Material). User Documentation #3 docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl enhancement [CDR-157] - docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl [GitHub issue] UI Navigation Customization - Can set the properties used to display/hide navigation tabs and the names of those tabs. (They are set using the /ced2ar-web/config page.) Administrator Documentation [CDR-157] - docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl [GitHub issue] Global Authentication Option Added - Setting accessMode to AdminOnly allows only users with the ROLE_ADMIN role to access the application. All others are prevented from accessing the pages. This is primarily used for codebook-development servers where crowd-sourced edits are curated and edited by administrators. Administrator Documentation [CDR-189] - Enable global authentication option in web properties Resolved Issues The following issues were fixed in this release. They are listed below. github Issues #26 Possible to delete investigator? #16 Variable edit mode: access levels question #9 Uploads sometimes fail and produce bad error message: invalid XML bug #8 can’t see crowdsourced edits; official version unviewable bug question #6 sha1sum bug #3 docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl enhancement jira Issues Issue [CDR-168] - Improve versioning automation [CDR-180] - ERROR SimpleAsyncUncaughtExceptionHandler...ArrayIndexOutOfBoundsException in ced2ar.eapi.VersionControl [CDR-183] - Configuration page is not updating [CDR-194] - delete investigator [#26] New Feature [CDR-157] - docDscr/citation/titlStmt/titl vs. stdyDscr/citation/titlStmt/titl [GitHub issue] [#3] [CDR-189] - Enable global authentication option in web properties QA Fix [CDR-181] - Version control is not commiting or pushing in v2 [CDR-191] - Enable generation of ced2ar.log [CDR-193] - header image for wiki-census

12. Replication Materials for Disclosure Limitation and Confidentality Protection in Linked Data

    Lars Vilhuber, John M. Abowd, and Ian M. Schmutte

    Dec 2017

    itemType: dataset

    DOI URL
13. Usage and outcomes of the Synthetic Data Server

    Lars Vilhuber and John Abowd

    2016

    URL
14. Replication data for: National estimates of gross employment and job flows from the Quarterly Workforce Indicators with demographic and industry detail

    John M. Abowd and Lars Vilhuber

    2014

    DOI URL

Other Publications