Design Challenge 1 Data Sets

September 19, 2021 (Last Modified: September 30, 2021)

This is a list of the data sets you may use for Design Challenge 1: One Dataset / Four Stories (Design Exercises 3 and 4).

You must choose one data set to create all of the visualizations in a phase. We strongly encourage you to use the same data set for phases 3 and 4.

For each data set, we have put the file in a Canvas file folder (the DC1 Data Sets folder). We will also create a starter workbook in Tableau Online, so you can copy the workbook (rather than uploading it yourself). See this Piazza Posting for instructions.

Note: for each data set, we’ve checked that it is in decent shape, and done some data cleaning. You may want to do more.

Also, for many of the sources, there is more data available. If you’d like to grab a different set from the same source, that is acceptable. You must choose a set that is at least comparable to what we provide. Please describe what you’ve done somewhere when you hand things in.

Also, note that this year’s list is different than prior years. (there are two that are the same)

1. Spotify

Note: this is a different (and bigger) Spotify dataset than we had in Phases 1 and 2.

The SpotifyFeatures.csv data set is a 32MB CSV file with 230K songs. For each song there are approximately a dozen features, as well as some identifying information.

The data set originally comes from Kaggle. There is some documentation for the features there, as well as in the documentation for the smaller dataset. Unfortunately, the Spotify Documentation provides little explanation of the features.

2. College Scorecard

This data comes from the U.S. Department of Education. This is a huge data set, that has a ton of information about all the degree granting institutions in the US. Some of the files have over 2600 features for each college. Most of these features are quite detailed, and relate to financial aspects.

If you want to get the data (be warned - there’s a 380M+ compressed zip file) and get more information you can download it from here. But, Abhay has processed the data to choose a reasonable subset. It should be more than enough to find interesting stories for this project.

Abhay’s files pick 37 features from the huge files, and put together multiple years. For each college, there is an entry for each year. You can get information about what the columns mean from the data dictionary.

There are two versions of the data: College_2012_2_2020.csv has years 2012-2020 in 2 year increments, while College_2000_5_2020.csv covers 2000-2020 in 5 year increments. You can choose either one.

One warning: this dataset has a bunch of “missing data” (empty entries that may show up as zeros).

3. Census Data by County

This data set aggregates many different quantities of interest over the counties of the US. In past assignments, we’ve provided even more detailed data sets which required even more aggregation.

The USDA provides this data as 4 separate sheets (on This page). Any one of them could tell an interesting story but together, they provide a very rich and complex data set full of stories.

We have (well, Young, the 765 TA in 2020 has) joined the 4 spreadsheets together (joining by the “FIPS Code” column) creating a single file. The four files are also combined (joined by the “FIPS Code” column) into one file, and put in Box. The rows for the states (not counties) are also removed. The data is downloaded and processed on September 13, 2020 from USDA/ERS.

The CSV File census_counties.csv is about 4MB and has 3196 rows and 339 columns)

The variable descriptions can be found on here, and some of them are replicated in the following table:

Variable Descriptions

Column name	Description
Births_2019	Births in period 7/1/2018 to 6/30/2019
CENSUS_2010_POP	4/1/2010 resident Census 2010 population
CI90LB017_2018	90% confidence interval lower bound of estimate of people age 0-17 in poverty 2018
CI90LB017P_2018	90% confidence interval lower bound of estimate of percent of people age 0-17 in poverty 2018
CI90LBINC_2018	90% confidence interval lower bound of estimate of median household income 2018
CI90UB017_2018	90% confidence interval upper bound of estimate of people age 0-17 in poverty 2018
CI90UB017P_2018	90% confidence interval upper bound of estimate of percent of people age 0-17 in poverty 2018
CI90UBINC_2018	90% confidence interval upper bound of estimate of median household income 2018
Civilian_labor_force_2018	Civilian labor force annual average, 2018
Deaths_2019	Deaths in period 7/1/2018 to 6/30/2019
DOMESTIC_MIG_2019	Net domestic migration in period 7/1/2018 to 6/30/2019
Economic_typology_2015	County economic types, 2015 edition
Employed_2019	Number employed annual average, 2019
ESTIMATES_BASE_2010	4/1/2010 resident total population estimates base
FIPS_Code	State-County FIPS Code
GQ_ESTIMATES_2019	7/1/2019 Group Quarters total population estimate
GQ_ESTIMATES_BASE_2010	4/1/2010 Group Quarters total population estimates base
INTERNATIONAL_MIG_2019	Net international migration in period 7/1/2018 to 6/30/2019
Med_HH_Income_Percent_of_State_Total_2019	County Household Median Income as a percent of the State Total Median Household Income, 2019
MEDHHINC_2018	Estimate of median household income 2018
Median_Household_Income_2019	Estimate of Median household Income, 2019
Metro_2013	Metro nonmetro dummy 0=Nonmetro 1=Metro (Based on 2013 OMB Metropolitan Area delineation)
N_POP_CHG_2019	Numeric Change in resident total population 7/1/2018 to 7/1/2019
NATURAL_INC_2019	Natural increase in period 7/1/2018 to 6/30/2019
NET_MIG_2019	Net migration in period 7/1/2018 to 6/30/2019
PCTPOV017_2018	Estimated percent of people age 0-17 in poverty 2018
POP_ESTIMATE_2019	7/1/2019 resident total population estimate
POV017_2018	Estimate of people age 0-17 in poverty 2018
R_death_2019	Death rate in period 7/1/2018 to 6/30/2019
R_DOMESTIC_MIG_2019	Net domestic migration rate in period 7/1/2018 to 6/30/2019
R_INTERNATIONAL_MIG_2019	Net international migration rate in period 7/1/2018 to 6/30/2019
R_NATURAL_INC_2019	Natural increase rate in period 7/1/2018 to 6/30/2019
R_NET_MIG_2019	Net migration rate in period 7/1/2018 to 6/30/2019
RESIDUAL_2019	Residual for period 7/1/2018 to 6/30/2019
Rural-urban_Continuum_Code_2013	Rural-urban Continuum Code, 2013
State	State Abbreviation
Unemployed_2019	Number unemployed annual average, 2019
Unemployment_rate_2019	Unemployment rate, 2019
Urban_Influence_Code_2013	Urban Influence Code, 2013

4. Time Usage Survey

The American Time Usage Survey (ATUS) tracks how people spend their time. There are corresponding international versions. There are actually lots of different surveys with interesting data available from the IPUMS website.

We have done a data pull for you. We are allowed to share a data pull ( see this). We chose to collect data from all of the years (2003-2019), and selected a wide range of different attributes. The data was downloaded on September 13, 2020 from ATUS.

Note: this is pre-pandemic data - we may try to create a new pull from the data set to get more updated information. I may make a newer data set, or a data set in a different form, available. Or, we may use that in a future assignment.

The CSV File atus_data.csv has 210587 rows and 34 columns.

You may create your own data pull if you’d like to try this with different columns (but be sure to document it in your writeup). Getting a data set requires picking from all the options - there are so many options that picking a good set is pretty time consuming. It is actually an interesting exercise to see how they document their data - they are very careful in documenting everything.

You can find out what the “time use codes” mean on this page.

Interpreting the other codes requires some digging, unfortunately. Some are self-explanatory, but others… I tracked down the “FAMINCOME” columns: explanation here. The state codes are here. Here are some of them are replicated in the following table: