Design and Development Project 1

Summary: This project will provide you an chance to practice your programming skills and practice with the agile software development methodology, continuous integration, automated testing and other more industry oriented practices that we don't often use in academia.


Agile: This project will be done in the agile methodology in that it will be done in a series of sprints. Each week there will be a 3-6 hour (estimated - though grads will have to spend longer on the project - perhaps closer to 5-9 hours) assignment which builds on the previous assignment. So be sure to be careful to practice good software development techniques so that you don't make life difficult for yourself later.


Individual work: Later in the semester we will work on group work. For this project you will work individually. You may ask your classmates for pointers on getting your git/github/jetbrains toolchain setup, but each student should be doing their own work. You can of course use my "as long as you are just talking and no one is on a recording device like a computer" guidelines to point each other at good libraries to do this work.


Tools: Use the the correct Jetbrains IDE for your language of choice (see below). If you have not yet done so, activate your jetbrains student account to get one year of academic use of the jetbrains professional development tools.


Version Control: In this class we will use git for version control.

We will use github for the first project (and likely the second as well). If you have not already done so, make a github account. It is free for the kind of stuff we are doing in class.


Programming languages: You may use one of several programming languages for this first project so long as you make it work on my machine (not just yours). With that in mind the following language are supported in this class.


Sprint 1: Getting some Jobs data

Due: one week after assignment.

Due: Monday Feb 3rd at 11:59pm

  1. Create a project in your jetbrain's IDE for your language of choice. Be sure to include your name in your project name (for example jsantoreJobsProject would be a fine name for me to use for this project. I have 70+ students doing some variation of this project and so I'll have a lot of problems with dozens of "project1" projects.
  2. Add the project to git using either the built in jetbrains tools or command line git
  3. share the project on github using either the built in jetbrains tools or command line git. Be sure to include any build files that I need.
  4. Add the appropriate github actions script to your github project, be sure to include a test runner and a linter.
    1. for example if you are using python you would use https://github.com/shinjitsu/introJavaFromPython/new/master?filename=.github%2Fworkflows%2Fpythonapp.yml&workflow_template=pythonapp The linter flake8 and the test runner pytest are already setup for you (if you want to use a formatter like python-black instead of a linter that is fine too)
    2. for example if you are using java you would start with https://github.com/shinjitsu/introJavaFromPython/new/master?filename=.github%2Fworkflows%2Fgradle.yml&workflow_template=gradle
       You will still need to add the test runner and linter, you will likely need to adjust the default jdk from java 8 (1.8) to java 11. See https://github.com/actions/setup-java andperhaps https://github.com/marketplace/actions/google-java-format (unlike python java have at least three widely used formatting guides)
    3. For Go programming language you could use the built in https://github.com/shinjitsu/introJavaFromPython/new/master?filename=.github%2Fworkflows%2Fgo.yml&workflow_template=go
    4. for other languages choose your best version to start with https://github.com/shinjitsu/introJavaFromPython/actions/new
  5. write a program which will (in at least three methods/functions so you can test properly)
    1. (undergrads only) use the github jobs API to retrieve all 226 (or how ever many there are by late January) For this first sprint, after your get each of the 5ish pages of jobs, put them into  a single collection (list/arraylist etc)
    2. Graduate Students Only: use the Hacker News API  Or the new firebase Hacker News API to Get the January 2020 ask HN who's hiring. Retrieve all of the top level comments from this article, parse them into job, location, required and recommended skills, visa sponsorship, onsite or remote and the longer job description and the website and store these data items in a list/arraylist etc of dictionaries/hashmaps/datacasses.
  6. For this first sprint, just save the data to a file
  7. Write a readme.md for your project in the github project (you can do it with your jetbrains IDE if you like)
    1. the readme should have
      1. your name
      2. any install and run directions I need beyond the basics.
      3. a brief description of what your project does
      4. a brief description of what is missing from the project (if anything)
  8. Write two automated tests
    1. one test should the method that retrieves the data from the web and assure that you get more than 100 data items
    2. The second test should check to see that the save file contains a job with a job title that you know should be there.
  9. make me a collaborator on the project
    1. to submit, make me a collaborator on your project. I'll git clone it, run it and the tests and linter, make an innocuous change to your project, and make sure the github actions work and grade based on these things.

You can see the grading sheet/rubric that I will use to see how the points are allocated.