Progress
So far for our project we have split off and started consildating the data from the different course catalogs into a data that we can then plot. Right now we are each working on taking that data and putting it into a CSV.
Problems
One problem that we ran into is the scope of the project and how much data we have. We changed the time period from every 10 years to every 20 years, so that we arent overwhelmed too much with the work. Additioanlly, another problem that we ran into is translating the data into CSV. We have decided many approaches to do this. The best approach that we came up with is do some of the work manually and some of it automatically.
Tools and Techniques
We have started to use automator on the Mac’s to turn the pdfs into a txt file to easily translate the data into CSVs. We also may forgo the use of Openrefine since most of the data we’re putting into the CSV is already being highly curated by us in the beginning.
Deliverables
Our goal this week is to get the CSV ready by Tuesdays and Wednesday’s and by Thursday and Friday we hope to translate taht data into a bar graph that will show the change of course structure.
Personal Messages:
Jamal: I have been working on cleaning up the dataset in order to see what data we can and can’t use.
Hadi: I have been working on cleaning the data for my assigned data set.
Feraidon: I have been working on cleaning the data for assigned data set.
Jonas: I have been working on creating a script to scrape the data. With this we can consoldaite the data into once CSV.
Gideon: I have also been working on cleaning up the data for my assigned course catalogs.
5 thoughts on “Final Project Update”
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
I’m exciting to see how you choose to visualize this data, did you find that the automator tool on Mac made any errors that you had to clean up?
That’s so cool that you found an automator tool on Mac that helps you put data into a CSV. I agree that your group probably won’t need to use OpenRefine for data cleaning since you guys are curating the CSV. I’m excited to see what you’ll end up doing with this data. There are so many options for this and I would love to see what details you’ll add to make this project your own. Overall, you guys seem to be making good progress.
Dealing with data can be a really tough task indeed, considering the variety of enrichment courses that Carleton offers each term. But I’m sure the results will be very interesting. Have you guys considered other ways of visualizing data besides bar graphs? Perhaps other methods could give us richer food for thought from a different perspective.
You guys seem to be working with the most data. While visualizing data seems to be the more challenging part of my group’s final project, for your project I imagine it is parsing and sorting through all the data, which requires a different, more technical skillset! I admire your efforts and am quite excited to see your choices on how to succinctly and effectively communicate so much information.
I’m also working on data cleaning and I think it really takes up time. Our data is in its raw text form and we want it to be counts eventually, and we also decided to do some work manually as we could not come up with a better way. I also agree that Openrefine may not be the best tool for the situation that you described. Looking forward to seeing your final project!