Project 1 Sprint 3

Summary: In this sprint3 of your first project, You will read data out of an excel file to add to your database


Tools:
We will continue to use the same version control, continuous integration tools and programming language requirements that we put together for parts one & two.

Sprint 3: Getting some additional data:

Due: Tuesday Feb 20th  at 11:59pm



Use the same project that you used last time. I will just do a git pull on the project and get the updates.

• Get the excel file  with additional jobs data that I pulled from another site.Some of the jobs seem to be posted to both sites, but not all of them
• the sheet with data is titled Comp490 Jobs. The first row is the column titles, the next 750 rows in the excel file are the jobs
• programmatically open the xcel spreadsheet and read the data out of the spreadsheet.
  
• Insert the data from the spreadsheet into the database. Make sure to get all of the important data (at least the information required in sprint 2 - except for the big job decriptions that aren't part of this data set). the two data sources will likely not use the same names, furthermore the data may not line up perfectly.
  
• Write automated tests
• write tests to assure your method to read from the xlsx file works properly
• eg, make sure you get data from multiple columns and that you read in at least 300 rows
• write a test to make sure the new data goes into the table
• write a test to make sure the old table is still there if test is still not there from previous sprint
• write a test to make sure that your new write to table function works (unless you found a way to make the old write to table work and the old test covers it all.
  
• make sure your DevOps stuff all works (tests are run on github and formatting checks still work)
• update your readme and requirements (or equivalent go.mod or gradle or whatever you need for your language.)
• commit and push often so you lose nothing if your computer dies.

As a preview, in Sprint4 you will have to look through all the jobs (from all three sprints) and and provide visualization for all of them as if they were one set of jobs.

For those of you who have not worked with excel files from a program before, I recommend you take a look a the following libraries (as long as it works on my machine, you can use other libraries, these are just suggestions.

Python:

I think openpyxl is still the most widely used library

https://openpyxl.readthedocs.io/en/stable/

https://realpython.com/openpyxl-excel-spreadsheets-python/

Go

In Go I've had good luck before with excelize

https://github.com/qax-os/excelize

https://www.kelche.co/blog/go/excel/

Kotlin

I've never tried it, but it looks like the most widely used library for this in Kotline is Apache POI (with or without the ExcelKt wrapper)

https://www.baeldung.com/kotlin/excel-read-write

https://github.com/evanrupert/ExcelKt

Submitting:

I will repull from the same repository after I wake up on Feb 21.