« Previous | Main | Next »

New BBC Comments Module: Technical Overview

Post categories:

Richard Summers | 21:07 UK time, Monday, 28 March 2011

I'm a developer of the Audience Publishing Services team in Programmes and On Demand in BBC Future Media. As Jessica has explained last week we rolled out a new comments product to all BBC blogs.

As a recent addition to the APS team I was tasked with revamping the commenting system and bringing it onto the BBC's Dynamic Web Application Platform (the Platform - PHP with BBC flavoured Zend framework).

The old system comprised of a set of API's which would allow developers to interact with the APS backend system. This would allow them to retrieve a list of comments (we call it a comment forum), and post comments back to it.

For ease of use, with products such as blogs, there was a set of SSI includes which would be embedded within a page which would pull in the necessary assets, forms etc and do all the API calls necessary to make it all work.

So why change?

The old architecture was basically a set of API's to the APS backend (which is tried & trusted), but the frontend implementation was difficult to implement and was totally geared towards working on the older SSI based servers, not the Platform.

We needed an easy way of adding comments to pages on BBC Online which would work on the Platform, would be backwards compatible with the older servers, and one that would stand up to the high traffic demands of clients such as BBC News and BBC Sport.

So what did we do?

Firstly, whatever system we built, it had to interface with the APS API (our source of comments and comment forums). We needed something on the Platform which would act as a wrapper for these API calls, so we build the Comments Service. The Comments Service is the bridge between the Platform and APS. Functions are called on the Platform using the Comments Service, which in turn calls the relevant APS API calls, which then returns JSON back to the Comments Service, to return it back to the application which called it initially.

It's not unusual for us to see peak levels of traffic of up to 500 transactions a second for News pages using Comments, so the Comments Service makes use of Memcached, this enables us to cache the response from APS for a short while, to relieve pressure on the APS Service at times of heavy load.

Graphs showing caching of bbc comments module on 21st March 2011

 

An important thing to note in the diagram above (taken from our trial of Comments on BBC News Stories), is the scale of the y-axis for each of the graphs. Even though the requests per second from the Comments Module to the Comments Service peaks at nearly 120 requests/second (requests for a list of comments), the requests to the Comments Service to the APS Service never goes above 10 requests a second. This enables us to cope with the large spikes in traffic and scale the module to meet the demanding needs of the bbc.co.uk domain.

The next thing we needed was something to handle the presentation of comments, the forms needed to interact with comment forums (posting a comment) and deliver the assets and javascript to enable a rich user experience for the whole thing. This, is called the Comments Module.

The Comments Module is a small application, which follows the design pattern of the BBC Module (others BBC Modules include Barlesque - the masthead and footer on all BBC pages). This allows it to be included in any of our Platform pages with only a few lines of code. To include the Comments Module on the old SSI based servers, we use the Comments Module as a full application, and call it via a dedicated URL using SSI. When the Comments Module is called in this way, it returns all the HTML, images and javascript needed to embed it into a page, so wherever the developer adds the SSI call to the URL of the Comments Module in their web page, the Comments Module will appear in all it's glory, already styled (in it's default appearance - which can be overridden with CSS), ready for use (including integration with our login system BBD iD).

The Comments Module uses the Comments Service to retrieve the comment forums from APS, and then renders them using the BBCs templating system - Spectrum. This takes care of rending the correct HTML, script, CSS tags etc.

So whats new in the new BBC Comments Module?

So, aside from a UI refresh (which at the moment is mainly visible on BBC News), we've added a few things which you might notice:

Speed Bumping

When this is enabled on a site, or comment forum, it enforces a minimum time interval between posts to the same comment forum by the same user. This is like a cooling off period between posts.

Ajax Pagination

When paginating between different pages of comments, if you have a javascript enabled browser with javascript turned on, all pagination happens using Ajax, making it load faster and increasing bandwidth efficiency.

Ajax Comment Posting

As with the Ajax pagination, posting comments can now occur using Ajax (specifically the BBC javascript framework Glow), meaning a faster response for users, and again a more efficient use of bandwidth.

BBC comments highlighted

Comment Highlighting

We found the old way of linking to a comment could be confusing, as in a long list of comments, it wasn't always obvious which one had been linked to, so we've introduced highlighting for linked comments. See here and here.

Comments Per Page

We've reduced the number of comments per page to 100 for all blogs (from 500). This reduces page size, which is better for bandwidth and page delivery times.

HTML in Comments

We allow a small set of tags to be used to style your comments on BBC Blogs. These are: <blockquote>, <em>, <li>, <p>, <pre>, <q>, <strong>, <ul>, <ol> and <b> These are not currently available for use on BBC News comments.

The Future

We've got a full roadmap for new features to be introduced in the coming months, i'm looking forward to telling you all about them, as we introduce them.

We're always interested to hear your thoughts, please use the comment box below!

Richard Summers is lead developer, User Publishing Services, Programmes and On Demand, BBC Future Media.

Comments

  • Comment number 1.

    One great thing I've noticed about the new comments module is that it responds to my browser's accept-language header when displaying month names. So when I'm using an Irish browser, the name of each month is automatically translated. I hope this is sign of things to come and you start doing it everywhere.

    However I have noticed a small bug, if I send a http accept-language header specifying Scottish Gaelic, the comments fail to load. Though this probably isn't a comments specific bug, I've noticed similar anomalies all around the bbc site (for example buzz). Not that I mind, I don't speak Scottish Gaelic.

  • Comment number 2.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 3.

    Before this new change, only the first ten entries in each monthly archive list had the number of comments shown against the entry, which wasn’t very helpful, but the effect of the new changes seems to be that the number of comments shown against each blog entry has now disappeared altogether (at least for pre-March 2011 listings), which is even worse.

    Russ

  • Comment number 4.

    On some early blogs, the comments are not numbered. Do you plan to retrospectively number them?

    Russ

  • Comment number 5.

    For a long page of comments (> 100) if I refresh the browser on the last page to make the last few new comments appear it returns to the first 100 comment page, particularly annoying if you are following a debate with multiple pages. Perhaps increase the resolution to 200 comments?

  • Comment number 6.

    Increasing the resolution won't help, unfortunately, marko. The underlying page architecture is wrong, and it always has been.

    Russ

  • Comment number 7.

    5. At 10:19am on 29th Mar 2011, _marko wrote:
    For a long page of comments (> 100) if I refresh the browser on the last page to make the last few new comments appear it returns to the first 100 comment page...
    ------------------------------------------------------------

    Just click on the time shown next to a recent comment and then refresh will take you back to that comment even if it is not on the first page.

    __________________________________________________________________

    It is not that surprising that the comments module gets to its busiest at lunch time. It would seem a good idea to schedule enough moderators to cover this busy period. Sometimes it seems they have gone to lunch when they are most needed and this delay to comments prevents the natural flow of ideas.

  • Comment number 8.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 9.

    Re your HTML in Comments section, could you please make an extended version of this a standard part of the blogging help pages?

    Since you started supporting the UTF-8 character set, there have also been differences in the HTML entities supported. A full list of supported characters would also be helpful.

    In particular, when the new look was first introduced, us bloggers could provide help for other bloggers by demonstrating HTML code. Right now that's impossible because we can post individual GT; (>) characters but not the LT; character.

    Inability to post the LT; character will be difficult on some technical/economic blogs and cause a lot of head-scratching by non-techies.

    Worse, it means that web techies like me can no longer help them!

    Previously we could demonstrate using the LT; and GT; entities. Is there a workaround?

  • Comment number 10.

    The comments module will always be busy at lunchtime and as someone mentioned in an earlier comment moderators should be out in force during this time (maybe they can schedule an early lunch (cant imagine that between 11 and 12 is too busy) :)

    Great post, its good to get an insight of how you are dealing with existing problems.

    Thanks,
    Scott McLay
    Editor of Scott McLay's SEO Blog

  • Comment number 11.

    Another bug that I've noticed is that the username at the top of the comments module (where it says "you are currently logged in as x") links to the wrong page for me.

    The usernames at the top of each comment link to the correct place.

  • Comment number 12.

    #11 lucas42
    "Another bug that I've noticed is that the username at the top of the comments module (where it says "you are currently logged in as x") links to the wrong page for me. The usernames at the top of each comment link to the correct place."

    Snap! I wasn't certain whether it applied to everyone or just to me because my username is no longer the same as my displayname, but it seems to.

    To clear any uncertainty, does your username/displayname at the beginning of the comments section just after the "You are currently signed in as" link to http://www.bbc.co.uk/blogs/profile.shtml?userid=27931 I wonder? If to the same userid, it's clearly a generic fault. If to a different one, maybe you too have a different username to your displayname.

    FWIW:
    ● The link to lucas42 (or You as you see it) is to http://www.bbc.co.uk/blogs/profile.shtml?userid=883812

    ● The link to Barbazenzero (or You as I see it) is to http://www.bbc.co.uk/blogs/profile.shtml?userid=12508774

  • Comment number 13.

    Richard,
    "We allow a small set of tags to be used to style your comments on BBC Blogs."

    I'm disappointed that you haven't passed by to reply to any of these comments, but live in hope.

    Your list of tags supported doesn't include the a tag, BTW.

    The blockquote tag didn't fail on the last version of the blog site, and nor I believe did the pre tag, but neither did they work, whilst the ul and ol gave odd results. As there is no publicly available test area, there follows a test of each...

    This text is within a blockquote tag.
    It should appear indented.


    This text is within a "pre" tag.
    It should appear in fixed-width font
    and allow multiple spacing between characters.


    The following is a test of the ul tag which should produce a bulleted list of li tags.

    • first

    • second

    • third



    The following is a test of the ol tag which should produce a numbered list of li tags.

    1. first

    2. second

    3. third



    In preview mode, all work except that the ol tag produces a bulleted list instead of a numbered one, but previous implementations differed in "preview" and "posted" modes, so we will have to look at the post-submit situation.

  • Comment number 14.

    PS to my #13 above, the pre tag is potentially very useful if you need show tabular columns, as the table tag has never been supported on BBC blogs, viz:


    Brighton Pavilion 2010
    Candidate Party Votes
    Lucas Grn 16,238
    Platts Lab 14,986
    Vere Con 12,275
    Millam L-D 7,159
    Carter UKIP 948
    Fyvie SLab 148
    Kara Cit 61
    Atreides Ind 19
    Total votes 51,834

  • Comment number 15.

    Preview and basic html tags are not loading or functioning on a number of blogs.

    Russ

  • Comment number 16.

    Hey guys, thanks so much for the feedback, it really is appreciated.

    Barbazenzero & lucas42 - thanks for pointing out the profile link bug, we should have a fix for that deployed in the next few days. Barbazenzero - you're also correct about the a tags, we do support an a tag, with an href attribute, and a title attribute. We are also working to address the ol tag issue.

    _marko - Yes the retention of the current page in paginated comments this is something we're addressing.

    And to Scott Mc and the rest of the contributors, thanks again for the comments and feedback!

    Rich

    P.S. #9 Barbazenzero - will get the GT; thing looked at, I know how useful that is, and appreciate techie help you guys provide to non-techie users. Rich.

  • Comment number 17.

    Richard,

    Thanks for the response. I will pass by again soon to see how it develops.

    The current version is the best so far at supporting usable HTML and it would be a pity not to sort out the "rough edges".

  • Comment number 18.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 19.

    Displaying the GT character is no problem in BBC blogs, but displaying the LT character still seems an impossible dream.

    Workarounds using HTML entities still fail, as do using the UTF-8 character numbers for LT, viz:

    Testing GT; should be & # 6 2 ; omitting the intervening spaces gives: >
    Testing LT; should be & # 6 0 ; omitting the intervening spaces gives:

  • Comment number 20.

    Has the non-operation of the basic html tags been fixed?

    Russ

  • Comment number 21.

    Russ,

    As far as I've seen the only tag which used to work but does no longer is the i tag, but the original em tag does the same job. Note that support is not guaranteed for the b tag, although at the time of writing it's still working view source. However, the original strong tag does the same job.

    What other tags did you have in mind?

  • Comment number 22.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 23.

    Thanks Barbazenzero, but I’m getting some strange behaviour. I can post plain text ok. A message with some html formatting appears fine in preview mode in this blog, but as soon as I try to post it, I get a message “We're having some problems posting your comment at the moment. Sorry. We're doing our best to fix it.” and the comment box locks up and becomes unavailable for further editing. (This message may or may not be associated with the presence of html components of course. It’s impossible to tell unless someone at the BBC tests to see what is going on.)

    Whilst the blockquote function resolves correctly in this blog preview mode, it does not resolve in other BBC blogs’ preview mode, e.g. Radio 4. However, in those other places, although the preview mode does not show formatting, the html-stripped comment does get through the posting phase.

    There therefore seems to be some inconsistency in how the preview and/or html scripts are loaded or handled across the blogs. Hence my original question.

    I will try a third attempt at putting in some formatting in a following post.

    Russ

  • Comment number 24.

    One of the problems here of course is having to wait for the non-operational 300 second counter to operate before being able to post a new comment. The result of being told to wait for 300 seconds is that the content of the comment box locks up and becomes unoperational as well.

    Really, it's total carp.

    Russ

  • Comment number 25.

    Nope, I can't post anything with html components at the moment. (I'm using Safari 5.0.4, but I can't believe the parser for this comment box is browser-specific. I might try something later in Internet Explorer to see whether that makes any difference.)

    Russ

  • Comment number 26.

    #25 Russ
    "I'm using Safari 5.0.4, but I can't believe the parser for this comment box is browser-specific. I might try something later in Internet Explorer to see whether that makes any difference."

    I'd find it hard to believe that it could be browser-specific. Posting HTML source code works fine for me in MSIE, Firefox & Chrome (from which I'm posting this message).

    Are you using an HTML editor? If so, you might have to open it in code view to strip out styles, fonts and the like, which can confuse the parser. Try the KISS principle of building up to complexity, and remember that the parser does add a br tag to the end of each physical line, so make sure you have line wrapping on.

  • Comment number 27.

    [Follow-up formatting attempt]: bold, italic using i, italic using em, and

    blockquote

    and an a ref (to the BBC front page).

    Russ

  • Comment number 28.

    Russ,
    Good try, but you're obviously having trouble with the a tag. See http://www.w3schools.com/tags/tag_a.asp

  • Comment number 29.

    Barbazenzero - no I'm using straight keyboard input, or pasted from a plain txt file, i.e. without styles, fonts etc.

    #27 seems to have worked, at least partially. What was confusing me was that the preview parser will accept a url without quotes, but the posting acceptance-stage parser will not, and will lock the comment box content and terminate the posting process.

    The a ref parser is a bit strange: the parser accepts both an

    a ref=

    opening tag as well as an
    a href=
    opening tag.

    (Not sure how the above paragraph will appear in post mode: the pre tags introduce an inconsistent number, either zero or one, opening/closing line breaks.)

    There seems to be no problem with the GT character >, either straight from the keyboard (shift period) or escape-coded, and whether pre or non-pre tagged.

    What I can't do is a LT character, either straight from the keyboard (shift comma) or by using escape codes. (See the last line of your comment #19, where it is also non-existent to me.) Further testing in the preview parser indicates that the LT character, whether straight from the keyboard or escape-coded, will not appear even if it is contained within pre tags.

    I'll need to do a couple of test posts in the R4 blog area to check the differences as reported in #23 are still present.

    Re-checking i tags, the remainder of this sentence should appear in italics.

    Russ

  • Comment number 30.

    Barbazenzero, yes, you're right, I should always be using a href format. (That wasn't the major problem actually - it was checking which convention for the url the BBC was now adopting. The preview parser was also putting me off, accepting any a tag immediately and putting it in bold, which I mistook for a correct parse.)

    I note i tags (which always worked previously in BBC blogs) no longer work for italics. I find the use of em and strong so archaic!

    Incidentally, it seems that any code the posting-stage parser doesn't like within a paragraph will hide any further content of that paragraph.

    Russ

  • Comment number 31.

    Is there a character or word count limit to comments?

    Russ

  • Comment number 32.

    Russ - comments on this blog don't have a character or word count limit.

    Thanks

  • Comment number 33.

    Thanks Nick, but I found I had to split a long comment into three parts yesterday on the Radio 4 blog - the preview screen and post button just would not work if the comment was input as a single entry.

    I found I also had to rewrite the blockquote content because the Radio 4 blog doesn't seem to like those tags.

    Russ

  • Comment number 34.

    This comment was removed because the moderators found it broke the house rules. Explain.

 

More from this blog...

BBC iD

Sign in

BBC navigation

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.