Project 1 Sprint 2

Summary: This multi part project will provide you an chance to practice your programming skills and practice with the agile software development methodology, continuous integration, automated testing and other more industry oriented practices that we don't often use in academia.

This time we will be saving our data into a database.

Tools:

We will continue to use the same version control, continuous integration tools and programming language requirements that we put together for part one.

Sprint 2: Saving the University data to a database

Due: Sunday Feb 14th at 11:59pm

Due: Wednesday Feb 17th at 11:59pm (note that this is a Monday schedule for Bridgewater)

(some stuff was brought forward from last time, it is highlighted in gray)

1. Use the same project that you used last time. I will just do a git pull on the project and get the updates.

2. Pragmatically setup the database if the table(s) that you need are not already there.

3. write a program which will, as before, with multiple methods/functions:

• 
  
  • use your API key to get college data, see this site: https://github.com/RTICWDT/open-data-maker/blob/master/API.md for documentation
  • you can find the possible field names for data in https://collegescorecard.ed.gov/assets/CollegeScorecardDataDictionary.xlsx
  • get the data for all universities with degrees of type 2 and 3
  • get the following fields at least:
  • 
    
    • school.name
    • school.city
    • 2018.student.size
    • 2017.student.size
    • 2017.earnings.3_yrs_after_completion.overall_count_over_poverty_line
    • 2016.repayment.3_yr_repayment.overall
  • 
    
  • MAKE SURE YOU get all the pages of data – by default it comes in 20 at a time. You need all 3000+ entries
  • Be sure to git commit and git push often so you don’t lose anything to a dead laptop.
  • 
    

4. Save all the data to the database!

5. Write two automated tests

• 
  
  • one test should the method that retrieves the data from the web and assure that you get more than 1000 data items
  • The second test should create a new empty database, run your table creation function/method, then run your save data to database method then check to see that the database contains the test university that you just put there.
• 
  

6. Update your Continuous Integration/devOps on github to run these tests

7. Update your readme.md for your project in the github project (you can do it with your jetbrains IDE if you like)

   1. the readme should have (if everything works this should be about a half page or less)

• 
  
  • your name
  • any install and run directions I need.
  • a brief description of what your project does
  • a very brief discussion of your database layout and the table(s) you used
  • a brief description of what is missing from the project (if anything)
• 
  

8. Commit whenever you have some small additional part that works.

   1. Use good commit messages which tell everyone (especially me as your supervisor or collaborator standin) what this commit does. Be sure to push after each commit so that you lose little to nothing if your computer dies.

   2. So push early and often.

If you want to min/max, here is the grading rubric/sheet/distribution