« Previous | Main | Next »

Taking the Proms to the semantic web in 10% time

Post categories:

Michael Smethurst Michael Smethurst | 17:25 UK time, Thursday, 4 September 2008

A few weeks back I had the good fortune to be handed a dump of a mysql database stuffed to the gills with historical Proms data. It's got every Prom from 1895 to 2007 (and (you'd hope) 2008 at some point).

I've started to wrap a Ruby on Rails app around it and the results can be seen here. For now it's pretty basic with lots of gaps left to fill but the intention is to link it to MusicBrainz / DBpedia and publish the results into the Linking Open Data cloud.

In the meantime I thought it might be interesting to develop the site in public so you can see all my mistakes in real time. So I'll be posting progress reports here and updating this post as things get added.

What's happened so far

I wrapped a very basic rails app over the data just to peek inside and check the relationships. The result was a simple domain model of people and works and events and etc.

The next step was to create a new rails app and begin to build the site one table at a time. The beginnings of the schema are here.

What happens next

The next step is to continue on, one table at a time through parts and events and performance and etc to fill out the site. And when that's done the interesting work of linking to external data sources can begin. At which point I'll probably be buying drinks for colleagues old and new.

With the data published and linked I'll hopefully manage to persuade boss type people to open it under a creative commons non-commercial licence. Add some RDF and the Proms can join Later, Top of the Pops and John Peel on the LOD cloud. Well that's the plan.

A beta (of a beta (of a beta...))

This is very much work in progress. Eventually the data should make it's way into /music (via the MusicBrainz next generation schema) and the upcoming /events. If we can match events to broadcasts some of it should also make it into /programmes.

Updates

  1. 05/09/2008

    Added works by genre: http://bbc-hackday.dyndns.org:2840/genres/8/works.

    Added genre for work: http://bbc-hackday.dyndns.org:2840/works/6196

  2. 06/09/2008

    Added works by commisssioners - http://bbc-hackday.dyndns.org:2840/commissioners/25/works

    Added commissioners of works - http://bbc-hackday.dyndns.org:2840/works/32204/commissioners

    Added works by composer - http://bbc-hackday.dyndns.org:2840/people/16/compositions

    Added composers by work - http://bbc-hackday.dyndns.org:2840/works/183/composers

  3. 09/09/2008

    Compositions by composer now only list compositions, not arrangements etc - http://localhost:3005/people/135/compositions

    Work lists now include composer - http://bbc-hackday.dyndns.org:2840/works/a-z/a

Comments

  • Comment number 1.

  • Comment number 2.

    Hi metanoize

    *none* of the links would have worked between 8pm and 9pm because I was redeploying the application. Apologies. Is back now.

    If you want to know when the app is down and when new features are added please follow @historicalproms on twitter:

    https://twitter.com/historicalproms/

  • Comment number 3.

    Michael,

    Congrats. It's an excellent achievement.

    But... you do yourself short with a typo on the homepage:

    "1985" should read 1895 surely?

    best,
    Trampus

  • Comment number 4.

    @trampus - thanks for spotting. is fixed now. must learn to type one day ;-)

  • Comment number 5.

    hi michael,
    by incident I also work in radio systems engineering (for orf.at, national public radio in Austria), and also program in rails around some database acces (radio drama, in my case). any chance that not only the concert data, but also your ruby code could be available in the public domain? btw, I really like incremental progress update, just like I program in rails (one table a time, adding associations).
    another question: from the table naming (plurals, alphabetically) I assume you use "HABTM" in contrast to "has many :trough" for the relationships, any reason to do this (as it seems has_many :through has no disadvantages but could offer the chance for extension fields to the relationships)?
    thank you so much for sharing publicly,
    karl.

  • Comment number 6.

    Hi Karl

    I intend to make the rails code available for download from the site. However, I'm not the world's greatest coder so I'm not sure it will contribute much to the world.

    On the associations question I tend not to use either HABTM or HMT. I've found that adding these associations tends to make me lazy in views and I end up with pages using 100 queries when one will do. I try to write a maximum of two queries a page: one to get the thing, one to get the things under the thing. Although I have broke this on work listing pages (http://bbc-hackday.dyndns.org:2840/works/a-z/a%29 where there's a separate query for each work to get the composer(s). Since a work can have many composers and each composer has many fields that make up their display name, I haven't been clever enough to work out a better query. It does make these pages very slow!

    Anyway, in general I like to be able to see my sql and find associations hide it and make me laz(y|ier).

    Hope this helps. Code will follow when the application has some semblance of shape.

    Michael

 

Copyright © 2015 BBC. The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.