Data Integration

Aim

The purpose of assignment 2 is to prepare a dataset with which you can conduct data analyses for the term project. This is related to the term project, but please note that this assignment should be individual work. Also, you need to explore some variables that can be used as a dependent variable and independent variables. More details will be explained below.

Dataset

For the project phase 1 report, you already selected a medical condition of your group's interest, and there should be more than 500 data instances. For the assignment 2, please think about your research question(s). With that being said, you need to explore variables in your dataset. You can change those variables later, but this assignment will help you better understand the dataset. Please visit the official website (Links to an external site.) and Github repository (Links to an external site.) and read the attached appendix document Download attached appendix document(p.1 – 13). MEPS data consists of various variables such as medical condition, socioeconomic factors (e.g., gender, region, race, and family income), and medical expenditure. MEPS data also consist of various files such as person-level (e.g., health status, demographics, and total $$ of care), event-level (e.g., healthcare service use), and condition-level (e.g., medical condition). For the full review of those variables, please look at codebooks (person-level (Links to an external site.)) and condition-level (Links to an external site.)). I also coded those variables (personal-level only) regarding usefulness for analysis (included vs. excluded, Heejun_Inclusion field) and variable type (independent vs. dependent, Heejun_Variable_Type field). You can find my version of the codebook from this link (Links to an external site.). Some dependent variables you can utilize includes but are not limited to:

• Total health expenditures
• Total inpatient expenditures
• Total emergency care expenditures
• Severity of Illness (attacks/year)
• Number of School Days Missed (Children)
• Number of Work Days Missed (Adult)

For now, just focus on personal-level (h129.csv) and condition-level (h128.csv) files for exploring variables. At least, find one dependent variable and five core independent variables that can be used for the group project by carefully looking at documents and websites above. The selected independent variables should not be randomly selected. They should be clinically related to your dependent variable.

The second step is to preprocess your dataset. Please utilize the "week6_example_code.ipynb" notebook for this process. You need little modification to complete data processing and integration. Select patients (h128.csv) with the your medical condition, drop attributes which are not necessary, and join the condition-level data (h128.csv) and personal-level data (h129.csv).

What to Do for the Assignment 2

1. Explore the dataset and find one dependent variable and five core independent variables at least
2. Explain what these variables are and how they are related to the medical condition of your interest
3. Preprocess your data. The preprocessing will include but are not limited to:
   • h128.csv
     • Select records that are related to your condition of interest by using ICD-9 code and drop duplicated records based on the 'DUPERSID' (rowwise selection)
   • h129.csv
     • Select attributes that are discriminative and/or meaningful (columnwise selection)
   • Join two files (h128.csv and h129.csv) after selecting patient instances with the medical condition of your interest

How to write

For the assignment 2, I will not put constraints on the format. However, you need to present, at least, one dependent variable and five core independent variables. Also, please explain what these variables are and how they are related to the medical condition of your interest. Up to this part, you can write the report in a MS word file. You also need to submit your Jupyter notebook you used for data preprocessing and the final data file (csv).

What to include

Your submission of the report should include:

• Your name
• EUID
• Title of the project
• What I asked you in the "How to Write"

Please attach a .docx file for the report and .ipynb and .csv files for the data processing. Again, this is an individual assignment.

Grading Criteria

Your reports will be evaluated on the quality of entries. By quality, I do not mean mechanics (e.g., spelling, grammar), but rather content. Your entries should demonstrate reflective and critical thinking, integration of materials and experiences, development of your own research questions, and general intellectual growth. Also, I will look at the data preprocessing steps you implemented.