FSMK data-v data visualization project [Road map]


#1

If you have noticed already I started a repository on FSMK’s GitHub account data-v approximately 3 months ago! This was around the time the [summer-camp][2] came to a close and we had a lot of data on our hands with respect to participants and volunters. We wanted to play around with this data, map these on a visualization platform like D3.js or Python’s blokeh and then gain insights from the visualizations. data-v, was thus born thanks to @Rahulkondi, @saky, Abraar and myself.

As you can see on the repo, there’s just one Python script that helps with fetching data into a format useful for visualization. A lot of ground has to be covered with regards to which platform to use, what insights can be drawn from the camp data, etc. Hope we can discuss these here and then transfer them on to the repo as issues. Looking forward to all of your suggestions.

I believe that this project would help us know the demography of both our volunteers and participants. It’ll aid designing future camps in a much better fashion. Also, contributors working on the project will pick up in-demand data viz skills.

PS: data-v is still young, lets grow with it!


#2

Hey!
Good work. I was just now checking the repository and look at script.
Can you state what more needs to be done? On what all kind of data can it be run on? What is the expected visualisation?
Thanks!


#3

Sample use cases:

  • Plot # of boys v/s # of girls
  • Plot the regions from where participants hail from
  • Analyze their feedback in terms of graphs and charts (satisfied or dissatisfied)
  • Plot skill sets of volunteers, which would help participants approach volunteers

Open for more!


#4

I’ll keep it simple.

To-do list:

  1. Clean data - check for inconsistencies and duplicates, remove them
  2. Might have to anonymize data before sharing on GitHub
  3. Determine insights that we can draw from available data & how theyll be of use to us
  4. Figure out a visualization platform - depending on comfort level of devs (D3, Blokeh, etc)
  5. Code away!

Kind of data - we have volunteer registration form data & camp registration form data available to us now. Feedback was collected after camp, I’m not sure how it is stored though.

Use cases are described in the second comment. :smile:


#5

Good good! Can you do the same for say our recent BBMP elections? You could showcase how many that got elected has how many cases against them and so on. I wanted to work on something similar. Got to know from Yogesh. But I am held up right now and unable to do it. Do you think it’s worth a shot?


#7

Where could we dig this data up from? It should be public, I’m guessing, but still.


#8

If nothing else, RTI?


#9

RTI will not solve the purpose. They will give us data in the form of Kannada pdfs, which is really difficult to make use of.

@shashankmc, Take a look at this - http://bbmp.gov.in/bbmp-election2. Data about the contestants is in the form of a PDF. (Kannada) It would involve a bigger effort to first convert PDF into text/html, translate them and then make use of it. For now, it’s not a viable option. Let’s focus on the data we have at hand.

I found another resource here which tells us winners. It’s in HTML - so we need to do some amount of web scraping to make the data usable (again an overhead). Our aim here is to start of small, play with available structured data and then scale/ deploy it to other solutions. I hope everyone can agree on this!


#10

This is the same problem with even #mapyourwaste data currently available with the civic authorities


#11

People at Janaagraha Centre for Citizenship and Democracy had got the data through RTI and digitized the same - https://groups.google.com/forum/#!topic/datameet/PG0q7JFg0nQ


#12

On a similar note, Sumandro’s rainfall maps based on IMD’s rainfall data developed using R - https://ajantriks.github.io/iwp/rainfall_maps.html


#13

Yes, I’ve heard about this.
I’m working on a similar project with eGov Foundation, Chennai. Data Kind volunteers.


#14

The first issue is out! Take a look at it and get hacking if you’re game. :sunglasses:

Issue: https://github.com/fsmk/data-v/issues/2

Mentor: @Rahulkondi


#15

This sounds quite promising. I have been working on an admin theme project, a NodeJS - ReactJS version of another popular open source project. It also contains beautiful charts and widgets for visualizing data. You can have a look at it here - https://github.com/booleanhunter/ReactJS-AdminLTE

Any opinions ? I’ll be working on this to make it complete, perhaps you could borrow few components from here if you plan on having ReactJS ?


#16

That looks wonderful. :smile:
The group has been inactive for a while. Everyone’s caught up with work and/or school.
But maybe you could take a session on a Sunday at the FSMK office and get more people interested in data-viz and maybe lead a project/activity. That would be a neat way to do it. We don’t want this initiative to die because we got too busy. Let me know if you need any help with that.


#17

Sounds good! Yes I’d love to take a session on a Sunday and explain how I started with making existing charts to be modular standalone components. Would really like it if more people get active and contribute back something.

I’ll post in the group when my schedule looks good.

Few months ago I had re-used a small part of this project on another one to show some visualizations - http://www.nammapolice.xyz/stats. This was for a hackathon, where me and a friend of mine made an android app, and we wanted to demonstrate a very basic analytics dashboard to show crime data in real time. You can see the complete project here - http://namma-police.github.io/


#18

@booleanhunter So sorry! I seem to have missed this.

Were you able to come down to the office to conduct a session? Let us know how we can collaborate to take this idea further. Nice viz!


#19

Oh hey - I’ve been busy for a while. Yes I’m thinking I’ll take a session sometime on the weekend, preferably this sunday or the next?