Integrating and extending content with Drupal

Django Beatty, Adub Drupal Solutions - http://adub.com

Case Study - IPC Ignite Digital

Integrating and extending content

with Drupal

I’d like to talk a little on how Drupal can be taken out of its typical setting and used to manage and extend content contained in other systems where a full migration isn’t viable.

http://adub.com

http://adub.com



Case study: IPC Ignite CMA

CMA = Content Management Application

IPC Media is one of the leading magazine publishers in the UK and a wholly owned subsidiary of Time Warner.

To illustrate this, I’d like to talk about a major project we worked on recently. This was a project to be used in one of IPC’s divisions to replace an existing CMA. I hadn’t heard of a CMA before but it stands for ‘content management application’. It’s not quite a content management system as most of us know them but it’s almost the same and I'll be coming to that shortly.

http://adub.com

http://adub.com



Enterprise considerations

One of the defining aspects that shaped this project was that it took place in an enterprise setting. This has some very unique characteristics which many Drupal projects don’t encounter.

Of course there’s traffic volume, scaling and performance. But operating in a large company also means that there is rarely such a thing as a greenfield project that exists in isolation. You have to adapt to company-wide security policies and deployment practices (which may limit access to databases or servers for example). You may also have to deal with issues of project scope as you may only be allowed to work on certain things for organisational or accounting reasons. And you may well also have dependencies on other internal (and external) systems.

In this case, we needed to deliver a system that formed a component of a larger existing architecture. Simply put, this was an integration project which used Drupal as a back-end application to push data to an existing database.

http://adub.com

http://adub.com



Split-architecture

• Content management application writes to database

• Site delivery application reads from database

Database

Content management

Site delivery

The architecture we were integrating to is something we describe as a ‘split-architecture’. In most CMS’s including Drupal, you have an integrated system where site delivery and administration is contained within a single application. But it’s not the only way. We inherited a different kind of system where there is a separate back-end application for content administration (the content management application), and another application which is dedicated to delivering a site. A database sits in the middle, with the content management app writing to (and reading from) it, and the site delivery app reading from it.

http://adub.com

http://adub.com



Project scope

• Content management application only

The scope of this project was to deal with the back-end content management side only.

http://adub.com

http://adub.com



1. Reasons why we used Drupal

• User system - Fully-featured and evolved

• Taxonomy - Simple but powerful

Drupal is probably not an obvious fit here to many people and we put a lot of thought into whether we should build a bespoke application or use Drupal (or even something else).

However, Drupal provided us with some very important functionality out of the box which affect not only cost but also time-to-market.

The user system has been well road-tested and has had many years of development and maintenance. If we were writing this system from scratch, we would have to emulate most of its features (e.g. lost password one-time login tokens). This would clearly be a project in itself and comes at a real cost to the business.

By taxonomy we usually mean categorisation and tagging of content. The taxonomy system has roots in academic Knowledge Management so it has a good theoretical underpinning, and is both simple while going beyond the capabilities of most CMS’s. It fitted our requirements very well.

http://adub.com

http://adub.com



2. Reasons why we used Drupal

• Modularity - Everything can be customised

• Open source - Maintenance and functionality

• Cost - What you put into it

The modularity of Drupal makes it easily extensible for our custom requirements now and in the future. The fact that this approach goes right through the core code is a key to understanding that Drupal can be seen as a framework as much as a product.

Drupal is a popular open source project which has been around for 8 years and has over 3000 contributed modules. This is important to us not so much for any ideological reasons but from a purely pragmatic business viewpoint. This means we are working with a publicly maintained infrastructure with dedicated security, QA and UX teams. And there is a strong network effect at play here (as evidenced by the huge ecosystem of modules). As a business we have a much better assurance of quality and maintenance because all the users have a stake and can influence its direction. For example, Drupal has strong multi-lingual capabilities because Sony BMG had a requirement for it. It’s clear that as Drupal becomes a standard platform for online publishing, the user-base will grow it to match their requirements.

Another reason was cost. As there are no licensing fees, using Drupal costs exactly what you choose to put into it.

http://adub.com

http://adub.com



Constraints

• Database must remain unchanged for site delivery application

• Phased delivery

• Must comply with corporate security policy

A few examples of the kind of thing we needed to accommodate.

As shown earlier, another application is making use of the data that our content management application stores. This means that the legacy database structure needed be kept.

A major part of the agile process in use at IPC is that projects must be delivered in finished usable phases. Thus the deliverables were easily divided into sections by content type (e.g. news, reviews etc.) so we could package delivery by site section.

Another constraint typical for enterprise is that we have to adapt to wider security policies across the company. At IPC, projects have to go through a pretty rigorous security audit by Time Warner. This is done after development and before a project can be deployed. When we started, we used the standard Drupal login which satisfied our policies. By the time it came to release however, these policies had changed. We found some available modules which did exactly what we needed, tested them, and solved this problem in a matter of a few hours. A good example of how Drupal’s flexibility can create business agility.

http://adub.com

http://adub.com



How we did it

So these were some of the challenges we were addressing and I’ll move on to what we actually did in this project.

http://adub.com

http://adub.com



Architecture

• Copy basic content to Drupal nodes

• Populate a connector table which bridges Drupal node id’s and legacy id’s

• Custom module per content type which handles legacy data

Essentially this was an integration project which had to maintain compatibility with a legacy database. Fortunately, Drupal’s hook system makes this relatively straightforward.

Our first task was to copy the basic content for each item into Drupal nodes (title, text, author, date) - so each item of content in the legacy database has a corresponding Drupal item. We then populated a bridge table which contained Drupal id’s and the corresponding id’s for the legacy data items.

Next we created a Drupal module for each content type. These modules used Drupal's hook system to keep the legacy database updated with any changes we made over on the Drupal side.

http://adub.com

http://adub.com



Drupal lets us extend legacy data

• Use of connector table allows us to integrate legacy and Drupal-native data e.g. Taxonomy

• ...and new modules can interact with legacy data e.g. Scheduler

So this was great - Drupal was enabling us to transparently manage data used on another system. But there’s an interesting corollary... The fact that we bridged the legacy data with Drupal allowed us to start extending it.

For example, there was no taxonomy system in the legacy data - so we used Drupal taxonomy straight out of the box. So if we want to introduce a new set of categories - for example, if we want to start associating our various types of content with movies - we can do this very simply with straight click-and-build and no coding required. And this applies equally to other new modules (for example, we can easily schedule publication/unpublication of stories using the Scheduler module).

Drupal has allowed us to add extra dimensions to our existing database and allow much richer applications of that data.

http://adub.com

http://adub.com



Data architecture

Bridge DrupalLegacy

Content Taxonomy Other new functionality

So to recap, here’s an overview diagram of our data. You can see that our legacy database still provides content for our site delivery app as before. But also Drupal is now extending that and providing a platform for new functionality to be added pretty easily. All back-end interaction is with Drupal, and this keeps the legacy database up-to-date and in sync.

http://adub.com

http://adub.com



Web services

But we found we could go a little further than this database approach with web services...

http://adub.com

http://adub.com



Drupal Services

• Services help us to further integrate Drupal back-end and site:

• Voting API

• Views

Drupal’s Services module provides a web services interface for our content management application. It's very full-featured, handling things like key management, authentication and providing various types of server for another program to connect to.

In order to allow rating of content on our site, we used the Voting API module via services. There’s a quick example coming up.

We are also starting to use Drupal Views directly in our site delivery, again accessed via services. This allows authorised users to quickly assemble views and get them onto the site.

http://adub.com

http://adub.com



Voting API via Services

Here’s a real-world example of Voting API being used to drive the Nuts ‘Rate My’ feature. When a user clicks on a rating, this triggers an AJAX call to services which registers the vote on our back-end application.

http://adub.com

http://adub.com



Images

• Custom crop for each image size

• Browser-based crop tool

Moving on from the overall architecture to some examples of custom workflow enhancements that were highly specific to the client. These modules aren’t suitable for release as they have dependencies on the in-house architecture. But it’s worth noting that there are now other ways to create similar interfaces suitable for general purpose use (e.g. using CCK, imagecache, draggable views etc.)

As these are big sites with dedicated picture staff, it was important to tailor very closely to their specific requirements and workflow. The main requirement that wasn’t answered by any existing modules at the time was to have different crops for each image size. We also wanted a browser-based crop tool.

http://adub.com

http://adub.com



Custom crop for each image size

As you can see, two crop sizes are shown - a portrait-shaped one and the smaller more horizontal one at the top called ‘Article’. Usually, the Article version of the image would either show a tiny scaled-down version of the image with big borders at the sides, or it would just center-crop and show the middle of the woman here. Clearly, what we really want in this situation is what we see here - a clear close-up of the face.

http://adub.com

http://adub.com



Browser-based crop tool

The user simply drags an outline. This is constrained by the proportions of the crop size and the user sees a preview below as they are dragging or moving the outline.

This process can only be done by hand as it depends on what is in the picture. However, when a picture is added, it initially gets cropped automatically - a picture editor can then recrop if it it doesn't fit their needs. We find this provides a pretty optimum workflow.

http://adub.com

http://adub.com



Image attachment

• Image thumbnail selector with search for attaching images

We also streamlined the process of attaching an image to a story. Out of the box, we had a huge drop-down select box of all image titles, which wasn’t really much use to us. What we needed was a visual selector. This also needed search and paging as we have a huge pool of images.

http://adub.com

http://adub.com



Image thumbnail selector with search

And here’s the result. It’s very straightforward to use. You can optionally add a search term. The results get paged (just out of the screenshot), and you just click on a thumbnail to change the attached image.

http://adub.com

http://adub.com



Galleries

• Creation of galleries using image search to select and drag-and-drop to reorder images within a gallery

Our galleries need to have captions for each image which only applies in the gallery context and images need to be orderable within a gallery.

We did a lot of work on improving the workflow of our galleries, including batch uploading of images from a desktop folder. We also created a very specific tool based around the workflow of our editors which allows fast creation of galleries.

http://adub.com

http://adub.com



Gallery creation

Here you can see how a gallery is created.

Images are selected on the right panel which allows the entire pool of images to be searched. A double-click on a thumbnail here will add it to the panel on the left which shows the gallery being created. On that side, an image can be double-clicked to remove and they can also be dragged around to reorder.

http://adub.com

http://adub.com

Integrating and extending content with Drupal

Technology

Transcript of Integrating and extending content with Drupal