Development Seminar Project 1

Due:

Feb  18th  at 11:59pm

Summary:

You will get the chance to investigate an api, read documentation, write a program and write tests for a program to get our initial creative juices flowing.

Details:

Stack OverFlow is as you all know one of the best places for getting tech help. They also have a very nice jobs posting list. You are going to query their jobs posting list and save that data to a database for processing in project 1.5

Nitty-gritty:

Write this program in either

• Java (target whatever version is on the university computers)
• or python (you can target any version of python 3.6+)
• golang
• rust
• (you can use c++ if and only if, you provide me with a configure script and a makefile for the project to make sure it works cross platform, something c++ is terrible at if you don't use a configure script and makefile)

If you really really really want to use a different language, talk to me, we'll see.

• write it to be cross platform on desktop machines, Python, rust and golang should do that anyway, Java tries to as well. But any external libraries need to work on linux especially.
• use one of the jetbrains IDEs to do your work. That should make the java work less painful in particular. We have them installed here at the university and you can get the full professional suite of tools for free as students so long as you are only using them for student work
• design your database to handle all the data you need. Notice that most people are pretty good about setting all the fields, but some people skip some of them. Have a principled decision about NULLs, will you allow them? will you fill in the violators with a default value?
• Save all of the stack over flow jobs into your database
• include the date that the job was added to your database.
• include at very least the job
• Title
• Company
• Location
• tags
• description
• salary if available
• onsite or remote if available.
• the update or publication date
• For this project you will need to use existing tools or you won't finish. We have talked a fair bit about the sqlite database, but you also need to use existing APIs/libraries to help you download the data.
• You will have an easier time parsing the jobs data if you use the rss feed.
• for python people, I suggest feedparser
• for java people, the rome library seems to be very popular
• Write a small suite of automated tests (lets say about 10ish assert statements worth) to test at least 3 different functions. Include these in your project
• When you are done, you need to write a short writeup (probably about 2-3 pages. Put this document in your project directory before you zip it up and submit. this document needs to include
• the list of dependencies for the project
• How you chose to implement the project and why
• Your data design for the project and why you designed your database this way.

Zip your entire project directory and submit the whole thing on blackboard.

Include in your zipped submission a file telling me how to run your tests using junit/pytest/built in testing (for golang or rust) which unit and functional tests show your code to work and be error free.