The agenda for Feb. 18, 2015:
With time shortened, I've decided to make a few alterations to the planned schedule.
- Will resume looking at machine-learning/data concepts next week, such as natural language processing with Python.
- The instructions for "Better Know a Former Congressmember with Grep" have yet to be finished though I will be significantly simplifying the assignment (including recording a screencast of one of the sections), because data-cleaning is such a pain-in-the-ass on its own and not within the full-scope of this class. So it has been reduced in value of points and amount of work, though I'll probably save it for another course.
- To make up for this, I've created a new assignment: Collecting Dallas Officer-Involved Shootings. The data-domain is a little more straightforward, though it requires you to practice your API-fetching skills and will make you think about data-publishing for a bit.
- Keep thinking about your final project, the difficulty and scope of which can be similar to the Dallas-OIS homework.
I haven't begun to cover the scope of all the command-line tools that might be helpful to you for a final project. But here's a few:
- tesseract - extract and process text from an image (i.e. scanned documents)
- pdftotext - we've used this before, and this is necessary when data/information is published inside a PDF
- avconv - a huge number of tools and methods for working with videos, including combining, splitting, trimming, gififying, colorizing, etc.
- convert - a huge number of tools to work with images. If you split up a video into image files via
avconv, then you can feed them into
montage program I demonstrated is from the same family of tools.
- Check out the documentation for the t Twitter tool to see the different kinds of data you can easily collect.