This assignment is designed to prepare you for the next phase of this class – data analysis and database design. Although you have been (inadvertently) using some of the ideas releva
This assignment is designed to prepare you for the next phase of this class – data analysis and database design. Although you have been (inadvertently) using some of the ideas relevant to this next phase, this exercise arrives at those concepts from a practical standpoint. It is also a great way to get you to start thinking about the term project that will be due by end of teh term.
While it is handy to know how to retrieve data from a table, combine data from multiple tables, summarize it before reporting, it is just as important to look at the raw data and try to understand what it is communicating. Your analysis and reporting will depend on the accuracy and completeness of the data.
Look at the data provided in the following CSV files and determine if there are any issues that will require attention. Specifically, look for:
- Missing data
- Unique data
- unique data identified above in other tables (you can JOIN!)
- Misspelled or incomplete data (male vs mael vs M)
- Outliers
TASK: Use features in Excel to spot these errors. Common techniques include sorting and filtering.Links to an external site. They are powerful enough to help you deal with missing and misspelled values as well as outliers. There are many other functions in Excel (see resource below) that can help you with understanding your data.
Resource: Excel functions. I used to link to a specific article but that link is now broken. I encourage you to use your preferred search engine and find sites you like that can help you – BingLinks to an external site., GoogleLinks to an external site., BraveLinks to an external site..
Data: VAERS data Download VAERS data(3 files in ZIP file) – UNZIP these into a folder first.
Submit (tips): A well-documented summary of the problems in the data linked above addressing items 1 – 5 that you were asked to look for. A grid like the one shown below may be used to organize your findings. You can do this in Google sheets or Excel and attach it here as a PDF using the headings shown below
Data fileColumn nameData issue observed (missing, misspelled, outlier, Unique)% of values with this issueImportance of addressing this issue (risk for ignoring). RecommendationOther observationsPreviousNext
VAERS_ID | SYMPTOM1 | SYMPTOMVERSION1 | SYMPTOM2 | SYMPTOMVERSION2 | SYMPTOM3 | SYMPTOMVERSION3 | SYMPTOM4 | SYMPTOMVERSION4 | SYMPTOM5 | SYMPTOMVERSION5 |
0164314 | Injection site erythema | 8.10 | Injection site induration | 8.10 | ||||||
0164315 | Hypoxia | 8.10 | Liver function test abnormal | 8.10 | Platelet count decreased | 8.10 | Pyrexia | 8.10 | Rash | 8.10 |
0164315 | Respiratory distress | 8.10 | ||||||||
0164316 | Asthma | 8.10 | Dyspnoea | 8.10 | ||||||
0164317 | Asthenia | 8.10 | Cerebral palsy | 8.10 | Strabismus | 8.10 | ||||
0164318 | Convulsion | 8.10 | Dyskinesia | 8.10 | Tremor | 8.10 | ||||
0164319 | Decreased appetite | 8.10 | Diarrhoea | 8.10 | Feeling hot | 8.10 | Irritability | 8.10 | Pain | 8.10 |
0164319 | Upper respiratory tract infection | 8.10 | Vomiting | 8.10 | ||||||
0164320 | Alopecia | 8.10 | Blood glucose increased | 8.10 | Condition aggravated | 8.10 | Diabetes mellitus | 8.10 | Ecchymosis | 8.10 |
0164320 | Eye oedema | 8.10 | Face oedema | 8.10 | Hypotonia | 8.10 | Visual disturbance | 8.10 | ||
0164321 | Arthralgia | 8.10 | Fatigue | 8.10 | Headache | 8.10 | ||||
0164322 | Irritability | 8.10 | ||||||||
0164323 | Bronchitis | 8.10 | Cough | 8.10 | Decreased appetite | 8.10 | Irritability | 8.10 | Laboratory test abnormal | 8.10 |
0164323 | Nasal congestion | 8.10 | ||||||||
0164324 | Injection site erythema | 8.10 | Rash papular | 8.10 | Rash pustular | 8.10 | Tenderness | 8.10 | ||
0164325 | Asthenia | 8.10 | Headache | 8.10 | Malaise | 8.10 | Myalgia | 8.10 | Pain | 8.10 |
0164326 | Dyspnoea | 8.10 | Fatigue | 8.10 | Hypersensitivity | 8.10 | Pruritus | 8.10 | Rash | 8.10 |
0164327 | Injection site reaction | 8.10 | Vertigo | 8.10 | ||||||
0164328 | Injection site erythema | 8.10 | Injection site hypersensitivity | 8.10 | Injection site swelling | 8.10 | Pruritus | 8.10 | ||
0164329 | Injection site erythema | 8.10 | Injection site swelling | 8.10 | Injection site warmth | 8.10 | ||||
0164330 | Injection site erythema | 8.10 | Injection site pain | 8.10 | Injection site swelling | 8.10 | ||||
0164331 | Fatigue | 8.10 | Pyrexia | 8.10 | ||||||
0164332 | Herpes zoster | 8.10 | Injection site pain | 8.10 | Injection site rash | 8.10 | ||||
0164333 | Erythema | 8.10 | Injection site erythema | 8.10 | Injection site oedema | 8.10 | ||||
0164334 | Injection site erythema | 8.10 | Injection site oedema | 8.10 | Injection site pain | 8.10 | Injection site warmth | 8.10 | ||
0164335 | Pyrexia | 8.10 | ||||||||
0164336 | Chills | 8.10 | Dizziness | 8.10 | Hyperhidrosis | 8.10 | Pain | 8.10 | ||
0164337 | Diarrhoea | 8.10 | Injection site pain | 8.10 | Myalgia | 8.10 | Pyrexia | 8.10 | Vomiting | 8.10 |
0164338 | Irritability | 8.10 | Oral intake reduced | 8.10 | Pyrexia | 8.10 | Rash morbilliform | 8.10 | ||
0164339 | Injection site erythema | 8.10 | Injection site induration | 8.10 | ||||||
0164341 | Dizziness | 8.10 | Migraine | 8.10 | Neck pain | 8.10 | Sneezing | 8.10 | ||
0164342 | Injection site swelling | 8.10 | ||||||||
0164343 | Injection site erythema | 8.10 | Urticaria | 8.10 | ||||||
0164344 | Rash macular | 8.10 | ||||||||
0164345 | Injection site erythema | 8.10 | Injection site swelling | 8.10 | Injection site warmth | 8.10 | ||||
0164346 | Hypersensitivity | 8.10 | Hypoaesthesia | 8.10 | Hypotension | 8.10 | Loss of consciousness | 8.10 | Tremor | 8.10 |
0164347 | Alopecia | 8.10 | Dizziness | 8.10 | Nausea | 8.10 | ||||
0164348 | Headache | 8.10 | Injection site erythema | 8.10 | Injection site induration | 8.10 | Injection site pain | 8.10 | Injection site warmth | 8.10 |
0164348 | Nausea | 8.10 | Pain | 8.10 | Pruritus | 8.10 | Pyrexia | 8.10 | Vomiting | 8.10 |
0164351 | Coma | 8.10 | Crying | 8.10 | Dyspnoea | 8.10 | Hypotonia | 8.10 | ||
0164352 | Convulsion | 8.10 | ||||||||
0164353 | Confusional state | 8.10 | Flushing | 8.10 | Hyperhidrosis | 8.10 | Hypertension | 8.10 | Memory impairment | 8.10 |
0164353 | Pallor | 8.10 | Respiratory failure | 8.10 | ||||||
0164354 | Guillain-Barre syndrome | 8.10 | Hypoaesthesia | 8.10 | Laboratory test abnormal | 8.10 | Muscular weakness | 8.10 | ||
0164355 | Drug ineffective | 8.10 | Hypoxia | 8.10 | Infection | 8.10 | Pneumonitis | 8.10 | Rash vesicular | 8.10 |
0164355 | Respiratory distress | 8.10 | Viral infection | 8.10 | ||||||
0164356 | Hypersensitivity | 8.10 | Injection site erythema | 8.10 | ||||||
0164361 | Abdominal pain | 8.10 | Cough | 8.10 | Diarrhoea | 8.10 | Headache | 8.10 | Influenza like illness | 8.10 |
0164361 | Myalgia | 8.10 | Nausea | 8.10 | Pharyngolaryngeal pain | 8.10 | Pyrexia | 8.10 | Vomiting | 8.10 |
0164362 | Convulsion | 8.10 | Dyskinesia | 8.10 | Eye movement disorder | 8.10 | Muscle twitching | 8.10 | ||
0164369 | Alanine aminotransferase increased | 8.10 | Alopecia | 8.10 | Antinuclear antibody positive | 8.10 | Arthralgia | 8.10 | Blood lactate dehydrogenase increased | 8.10 |
0164369 | Blood urea increased | 8.10 | Depression | 8.10 | Dizziness | 8.10 | Ecchymosis | 8.10 | Fatigue | 8.10 |
0164369 | Headache | 8.10 | Immunoglobulins increased | 8.10 | Infection | 8.10 | Insomnia | 8.10 | Malaise | 8.10 |
0164369 | Nausea | 8.10 | Purpura | 8.10 | Rheumatoid arthritis | 8.10 | Skin disorder | 8.10 | Spine malformation | 8.10 |
0164369 | Urticaria | 8.10 | Vasculitis | 8.10 | Viral infection | 8.10 | Weight increased | 8.10 | ||
0164375 | Eye discharge | 8.10 | Irritability | 8.10 | Lethargy | 8.10 | Pyrexia | 8.10 | Somnolence | 8.10 |
0164376 | Arthralgia | 8.10 | Carpal tunnel syndrome | 8.10 | Headache | 8.10 | Myalgia | 8.10 | Nausea | 8.10 |
0164376 | Pyrexia | 8.10 | Tendonitis | 8.10 | ||||||
0164377 | Pyrexia | 8.10 | Rash | 8.10 | ||||||
0164378 | Crying | 8.10 | Dyspnoea | 8.10 | Flushing | 8.10 | Pyrexia | 8.10 | ||
0164379 | Rash | 8.10 | ||||||||
0164380 | Rash | 8.10 | ||||||||
0164381 | Injection site erythema | 8.10 | Injection site mass | 8.10 | ||||||
0164382 | Injection site erythema | 8.10 | Injection site swelling | 8.10 | Screaming | 8.10 | ||||
0164383 | Arthritis | 8.10 | Bone disorder | 8.10 | Joint swelling | 8.10 | Pain | 8.10 | ||
0164385 | Pyrexia | 8.10 | Rash | 8.10 | ||||||
0164386 | Fatigue | 8.10 | Gait disturbance | 8.10 | Muscle spasticity | 8.10 | Paraesthesia | 8.10 | ||
0164387 | Arthralgia | 8.10 | Disturbance in attention | 8.10 | Hypercholesterolaemia | 8.10 | Memory impairment | 8.10 | Myalgia | 8.10 |
0164387 | Pneumonia | 8.10 | ||||||||
0164389 | Urticaria | 8.10 | ||||||||
0164390 | Injection site erythema | 8.10 | Injection site swelling | 8.10 | Injection site warmth | 8.10 | ||||
0164391 | Injection site pain | 8.10 | ||||||||
0164392 | Hypoaesthesia | 8.10 | Rash | 8.10 | ||||||
0164393 | Erythema | 8.10 | Mastitis | 8.10 | Pain | 8.10 | Pyrexia | 8.10 | ||
0164394 | Dizziness | 8.10 | Dyspnoea | 8.10 | Nausea | 8.10 | Syncope | 8.10 | Visual disturbance | 8.10 |
0164395 | Cough | 8.10 | Lymphadenopathy | 8.10 | Oedema | 8.10 | Pharyngolaryngeal pain | 8.10 | ||
0164396 | Urticaria | 8.10 | ||||||||
0164397 | Dyspnoea | 8.10 | Eye oedema | 8.10 | Lacrimation increased | 8.10 | Ocular hyperaemia | 8.10 | Pain | 8.10 |
0164397 | Pharyngeal oedema | 8.10 | Pharyngolaryngeal pain | 8.10 | ||||||
0164398 | Hypoaesthesia | 8.10 | Pain | 8.10 | ||||||
0164399 | Hyperhidrosis | 8.10 | Hypoaesthesia | 8.10 | Palpitations | 8.10 | Syncope | 8.10 | Tremor | 8.10 |
0164400 | Ear disorder | 8.10 | Oedema | 8.10 | Urticaria | 8.10 |