Crowd Sourcing Art Descriptions: Your Paintings Tagger

At the moment I am a full-time student in City University studying Information Science. As part of that course we learn all about information resources and how they are organised. We are taught the importance of categorising and organising information resources in a way that makes them are useable. What this means is making things easier to find. As you can imagine this is no small feat as the creation, storage and dissemination of information is growing at a rapid pace. Many of the people who are charged with organising information and making it easy to find are increasingly turning to automated systems to cope with the sheer volume of material. Unfortunately  if we cannot find the information it may as well not exist, so it is imperative to get this right.

This leads me onto art. Images are notoriously difficult to index due to their nature. Computers really struggle with automatically indexing and describing them. Right now they can maybe get colours or shape, but meaning is so far out of its reach. This requires people. However people require time and resources to be able to sit down, go through the materials and figure things out. But what happens when you have 200,000 oil paintings that need describing so that the owners, members of the public, can access them? I cannot even fathom how long that would take if a small team were tasked with sorting this mountain of material.

The Public Catalogue Foundation and the BBC have come up with a very clever idea to do this. They have created the My Paintings Tagger, a system where members of the public are called in to describe what they see, using a mixture of controlled (things such as landscape, portrait etc.) and uncontrolled (type as you like) terms. They have the paintings loaded up with the names, years and galleries, so all volunteers need to do is to describe what they are seeing. This is a wonderful tool, as a library student it gives me practical experience of describing resources, and as an a huge art fan I get to see paintings from all over the UK from the comfort of my own home. I really learned a lot about resource description some of those lessons I will discuss now.

1. Subjectivity

This process really reenforced the idea that when describing what you see, it is very subjective. Take for example the painting Earth II by Garry Marshall. It is a beautiful piece but how on earth (last pun I promise) do you describe what you are seeing? Black lines with brown? Or do you decide that it is a wasteland/coast/road etc. and go with that? But it is fair to stamp your impression on it when others may feel differently about what us being shown.

Earth II by Garry Marshall

Earth II by Garry Marshall

Things are so much easier when you have an image of something depicting a well known subject. See for example The Virgin with the Standing Child by Bernardino Luini. We know who is in the picture, she has a book and is wearing a cloak. This is also a good example of how context changes how we see an image and adds meaning. Automatic systems may see a child and a woman, but a human can give context through their recognition of the pair as Jesus Christ and Mary.

The Virgin with the Standing Child by Bernardino Luini

But even when you do recognise something and it has a shape and a form, this does not necessarily make it easy to describe. Take a landscape paining for example, are those green things trees, bushes, shrubs or something else? I see people but what are they doing and how should I describe that in a way that people can use to retrieve the image?

2. What is the point of this painting?

Linked to the issue of subjectivity, one element I really struggled with was selecting the subject of a painting. Take Putney Bridge and Village from Fulham, London. What is the subject of this painting? Is it a riverscape, a cityscape, a picture of industry, a study of people etc? Or is it all of these things, and then you have to decide how to allocate a subject? What I have learned through this process is that often there is no correct answer, so just go with what you would use if you were searching for this picture. If you do this then I figure you cannot go far wrong!

Putney Bridge and Village from Fulham, London by British School

3. You can lose everyday words

I like to think that I have a pretty good vocabulary. It isn’t the most diverse in the world, but I feel that I could point at most everyday objects and scenarios and be able to describe them. Hence I am building a career in information management. Turns out that my words are pretty rusty. There were many times during this process where I looked at things that were in the pictures, some of which I had seen before, but could not for the life of me, describe/name what they were. It was the most frustrating of processes. However the end result was that I racked my brains (or quit and went to Google) and became reacquainted with the terms. So through this process of using what I know to describe images, the images taught me back. Except for this one, Menton and La Tête de Chien, France, from the Frontier by William Logsdail. Can anyone please tell me if there is a special name for the wall/fence thing or is it just a wall/fence?  I thought it looked too fancy and must have it’s own name. For now I remain ignorant.

Menton and La Tête de Chien, France, from the Frontier by William Logsdail

Through this process of “using my words” I came to realise just how much slang/lazy language I use. Their are too many “thingys” “yokies” and “thingamajigs”. Take Side Elevation 1 by Prunella Clough. After I mastered the art of “triangle” and “line”, all I had left to describe this lovely painting was “pink squiggle”. Practicing how to describe abstract shapes will be next on my learning-to-describe-things agenda (except there I go and use the word “things” – I need more practice).

Side Elevation 1 by Prunella Clough

4. Simplicity is not a trap

After my mental gymnastics the painting Bendigo by Henry Frederick Lucas Lucas, popped up on the screen. I froze. This painting was a horse, standing on some hay. That was it. There was nothing else to see. Even the wall behind the horse was plain grey and I could not tell if the horse was outside, in a stable, etc. It seemed so simple that I almost skipped it. But I did not and having considered “Bendigo” I learned a valuable lesson; sometimes if things look simple, they often are. Not every image has to be a complicate mix of striving for a perfect word or weighing up retrieval tactics. Sometimes all the resource needs is a few simple words.

Bendigo by Henry Frederick Lucas Lucas

Once I had finished with Bendigo I had to laugh and I did wonder if the random painting generator had become self aware with a sense of humour. After all my fretting about this simple portrait of a horse, the very next painting that I received was Two Horses in a Paddock by George Stubbs. Double my strife!

Two Horses in a Paddock by George Stubbs

5. It is fun!

Some of you may have reached with point, with a raised eyebrow wondering if I should get a hobby or leave the house. But trust me, this is fun! The creators of the website have very cleverly use gamification in the process. Based on how many paintings you complete, you get a different coloured paintbrush. When you “level up” there is an explantation on the history of colour and its use in art. Really interesting and very smart.

Also every now and then a message will pop up for a museum or gallery thanking you for your time and effort. So you get to feel all warm inside as you are helping out these great cultural institutions. The actions are small but when everyone contributes a small piece it can make a big difference.

So there you have it and what are you waiting for? Get over to Your Painting Tagger and give it a go for yourself. If you do, let you know how you get on!


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>