TwaperKeeper archives

TwapperKeeper is shutting down it seems. It’s a popular online tool for archiving hashtags and other twitter searchers, and certainly well used in UK Higher Education where I work. I actually met John O’Brien, founder of TwapperKeeper, when he attended Dev8D (a developer event for those working in UK HE) a couple of years a go, nice chap.

Anyway, Martin Hawksey has created a wonderful tool for archiving, ummm, archives before they are gone. The tool is actually a Google Spreadsheet and to me it’s a testament to Google Docs power that an application that fetches and stores data from another site can be created using it.

Here’s my Twappers. Saved thanks to Martin’s brilliant spreadsheet.

  • UKSG is an organisation whose members are mainly University Libraries and Publishers. While I only attended their conference for the first time this year, it seems I was the person who originally created a TwapperKeeper archive, you can access the document by clicking the link (and if you’re signed in to Google Docs you can save a copy etc), note you need to click the archive tab at the bottom.
  • data.gov.uk – again nothing really to do with me, but an archive of tweets mentioning data.gov.uk (again, click the archive worksheet tab at the bottom)
  • bcb4 – Barcamp Brighton 4. I attended this event a couple of years a go
  • dilsr – Developing Innovative Library Support for Researchers. Not only did the twitter archives of this event, held at Sussex last year, almost disappear, but the website originally on ning has already gone with the dodo.
  • nickcleggsfault – during the run up to the election the Lib Dems looks like they may come a respectable third place rather than a distant third place. Our great papers put their usual impartial views to one side to – for the sake of Britain – destroy the Lib Dem leader. Twitter decided to join in. (I blogged about some of the articles for one day at the time). For some reason, I can only get 4,500 tweets, I think a tweet around that point is causing an error, I will try to get more.

I make no claim on owning any of the data. I’m guessing the original tweeters do. Or maybe Twitter Inc. Or Facebook. Actually it’s definitely Facebook. And it’s already alerted your mum that you’re reading this. Sorry.

Personal URL shortener

Back in 2009 I wrote a post about creating my own url shortener. I didn’t get very far, partly due to the challenge of how to create the short url codes, and partly due to my development philosophy of having an idea, musing about it, and never getting anywhere with it. Though I did install some code Ben Charlton has written and set it up as u.nostuff.org.

Part of the drive to do this was a bizarre feeling of guilt when using ‘short urls’ from the main shortener services. Each short url I created would mean we – as a userbase – were one step closer to the URLs being one digit longer (i.e. when that service ran out of free letter/number combinations with the current number of digits). Short urls were a scarce good. Did I really need this short URL? Worse, useful services such as twitterfeed use a short url each time they add something to twitter, even if most of those links would never be followed.

However there should be no need for all of us to be using the same small set of url shorteners, if we used our own, or one for a particular group of people, or for a particular service, then we would be free to use them as we pleased. What we (internet users at large, and especially twitter users) needed was lots of URL shorteners, each able to produce short codes.

Back in April this year I came across YOURLS, an open source simple URL shortener you can install on to your site [main website | blog with new releases | Google code]. This fitted the bill exactly. Installing it was easy on my dreamhost account.

Now I just needed the short domain name. I came up with a few catchy/clever domain names, but quickly found that two digit domains were pricey or just not allowed for purchasing. especially those such as .im, ly etc.

It would have to be three digits, and if I couldn’t get or afford ‘catchy’ why not just go for something completely random? After all I just needed to remember it.

I also decided that I would buy from my existing domain name provider, rather than signing up with another organisation, especailly some of the dodgy looking ones providing sole access to unusual country top level domain names.. I also wanted to use a country that was stable and unlikely to be awkward with renewing the domain or changing the rules (sn.im didn’t work for a while while the owners argued with the Isle of Man registrar). Using 123reg, at the time the cheapest three digit domains were to be found  (amongst a couple of others) in Belgium, .be. The only concern being as the two halves of Belgium seem to drift further apart from each other, there is even talk of one day Belgium splitting up (and I bet they wouldn’t even think if my domain name during the split, well I really).

So I ended up with xd5.be. No good reason. I used a combination with an x as I figured fewer organisations will have an acronym with an x in it, the number in the name may help in that respect as well.

The solution works well, with the usual browser bookmarklet (I’ve always shortened urls before pasting them to a twitter client). One thing I did find was that I was reluctant to use it at first as I was aware I was creating the shortest urls available on this domain (i.e. one digit code) and felt the need to preserve them for stuff that was important. Now I am on to two digits I am more care free! YOURLS allows you to choose between just using lowercase or including uppercase digits as well, I went for the former while the latter will obviously provide far more combinations of characters. I could also install additional copies of the software on to subdomains, e.g. a.xd5.be b.xd5.be, these are still seven characters in total, four less than tinyurl.com. drop me a line if you would like one of these.

In all, I think it is worth doing, it keeps you in control of your links and the software you are using.

Twitter clients

From about an hour after signing up to Twitter until very recently I used Twirl on both PC and Mac as my Twitter client. I was happy with it, and still am, but had noticed people using other clients and wanted to see if I was missing anything.

I round up my findings here:

Picture 1.png
Three twitter clients

Continue reading

People power : twitter is highlighting & affecting important issues.

A few weeks a go (why, the 12th Oct in fact) I was sitting at my laptop during the evening, doing this and that with Twitter ticking away on the right. I glanced at the newest tweets to pop in and noticed one from secretlondon.

200911121603.jpg

Curious, I read the Guardian article it linked to. A gagging order to stop a paper report the proceedings of parliament. This is not very good. I muttered and got back on with the this and that. A few minutes later another tweet from secretlondon came in:

200911121605.jpg

Now we have seeds of information! To Hansard, To Wikipedia, To Google. Who were Trafigura, who were Carter-Ruck?

Soon other tweets were coming along about this, and I was adding my two pence too, re-tweeting the news and adding my own little links to what I was finding.

Hansard provided the details the Guardian couldn’t report, and it quickly became clear what they were trying to hide.

By now twitter was alight. Hashtags came in to usage. Following these produced more information, once someone found something, they didn’t just share with their followers, but with everyone now following those tags. Previous Guardian articles (amongst others) were brought to our collective attention.

Before this I had not heard of Trafigura or Carter-Ruck. I suspect many were the same, yet now we were angry about what we read about their questionable activities (one apparently dumps nasty stuff in Africa, the other boasts about suppressing the press, regardless of truth). A storm was brewing and I felt it had yet to peek. But it was late and sleep beckoned.

The next morning I was curious if there had been any developments over night.

First thing I came across was a Spectator online article (a publication on the other side of the political spectrum to the Guardian). It quoted the Guardian article heavily, but then went on to quote the part of Hansard that contained the question (and company name) that the Guardian could not and provided links. I tip my hat to them. #Trafigura was now trending, celebrity twitterers (including our Lord Steven Fry) were highlighting it and more.

It felt like it was everywhere, on the news, and over in coffee room colleagues were talking about it. The Streisand effect had truly kicked in. Before noon on the 13th the case had been dropped. The Guardian was no longer prevented on reporting on the story.

Five Days later

Five days later a Daily Mail columnist Jan Moir wrote a homophobic piece (since edited) about the very recent death of singer Stephen Gately. A similar thing happens. A storm brews up. Not organised. But distributed little efforts or raising attention, mainly through twitter, which leads to coverage in main stream media and changes to the article and headline.

Black Out and Breaking News

And back in February there was a ‘black out’ campaign because of a proposed repressive internet law in New Zealand. Again, partly due to the coverage on sites like twitter, the section in question was scrapped.

It’s not just campaigns and activism. Breaking news is spread rapidly via twitter, such as the plane crash in New York (Twitter broke the story, and first images of the plane in the water came from Twitter), and Michael Jackson’s death.

But twitter doesn’t always have it’s own way. There was a Green avatar campaign for democracy in Iran, which sadly never saw success.

We’re seeing two things here…

1 – That information is now able to spread much faster than it ever has before. This has always been the case with the Internet, and has increased each year with new technologies (blogs, social networking), but especially with twitter.

2 – That people spreading this information leads to the main stream press reporting on it, and those under pressure back tracking. (I wonder how essential that middle step is?)

Twitter is such a good tool for the first point. It is instantaneous, not just the web, but on computer twitter clients and phones, and messages are public by default (unlike many other social networking sites where they are restricted to a specific groups or trusted circle of friends). Having an Open technical platform (which allows any other website or application to access tweets) also helps.

So…

My instant reaction is that this must be a good thing. When something bad is happening in the world (sorry, that sounds very simplistic) twitter, and other websites, can spread the information quickly and widely, even to those who don’t follow the news each day. This can lead to positive change.

The Trafigura/Carter-Ruck case is a good example of this. Imagine if it had happened 10 years a go. People (well, only Guardian readers) would have read the Guardian front page but not had a clue what it was about. In fact The Guardian may well not have run it as a front page story (or at all) as it would have simply confused/frustrated their readership. The Guardian took a gamble by putting this on to the front page, knowing (hoping) it would then become a story in its own right. It did, probably more than they ever hoped.

Noteworthy information is a virus, once it is in the wild it is unstoppable.

But all is not rosy. It will be slippery slope. I’m reminded of the 1995 film ‘the Last Supper‘. In the film they start off killing of the worst people in society, but as time goes on, things become more complex and grey and less clear cut. The Trafigura case was clear cut. Those trying to stop the BBC putting Nick Griffin on to Question Time, less so. There’s a thin line between the people power righting wrongs and mob rule.

One final example – baby and bump

Last Christmas I came across a news story about a ‘Lapland in the New Forest’. Long story short it was a con. Promised a lot, but was little more than muddy fields and a few fun fair (pay to use) rides, two santas (queue for hours, not allowed to take photos) and the odd tree with fairly lights, with staff who were untrained and the worst possible people to be interacting with kids.

For some reason I looked up to find out more. And I came across a thread on a web based forum called baby and bump (you can guess what it’s for). The thread was the top result on Google so became one of the main exchanges on the web for those affected by this.

The thread starts off with a few excited people discussing going to the Lapland attraction and how excited the kids are, and how much they have splashed out (money they couldn’t afford to throw away). Then those who visited the first few days after opening reported back, while others are in denial that it can be that bad. Then it really starts, more report back, and others start to join the forum simply to add their experience.

Then the fact finding starts: the owners name and address, other business addresses, legal rights, who in the council to complain to, who in the press to contact, how to file a small claims, the owner is related to the leader of Brighton (my) Council!

I like this example. It isn’t the twitterati or tech-savy web2.0 types, but just families on a simple web forum. No one organised anything, but many added bits of info, supported others, or shared their experiences. I would say it very much played it’s part in the early closure of this cruel con. After returning from a horrible day, cold, upset kids, after paying quite a sum upfront, it must feel frustrating and helpless, I think even finding others who have been through the same must be of some help. The Internet can really help in such situations. But it’s not the power of the internet, it’s the power of people. The Internet just acts as a enabling tool.

So Twitter is allowing us to share information and become aware of facts/situations in a way not thinkable until now, and at a very quick speed.

JISC, Monitter and DIUS (Department of Innovation, Universities and Skills)

Earlier this week the Jisc 2009 Conference went ahead. A one day summary of where things are going in Jisc-land.

Like last year, I got a good feel of the day via twitter. I used a web app called monitter.com for real time updates from anyone on twitter who used the tag #jisc09. monitter.com allows you to track a number (3 columns by default) of words/searches, this works well as these can be usernames, tags or just a phase. I used ‘jisc09’, ‘brighton OR sussex’ and ‘library’.

The keynote talks was also streamed live on the web, the quality was excellent. Check out the main Jisc blog for the event.

Linking to all the different sites, searches and resources on the web after the event wouldn’t do it justice. The usefulness was in the way these were all being published during the day itself, using things like twitter (and bespoke sites) as a discovery mechanism for all these different things being added around the web. I didn’t know who most of the people were, but I was finding their contributions. That’s good.

An email came out the next day about the conference and announcing a guest blog post by David Lammy, the Minister for Higher Education, on the Jisc Blog.

He finished by asking for the conversation to continue, specifically on  http://www.yoosk.com/dius which is described as ‘a place to open up lines of communication between Ministers and the HE Community’. Yoosk.com is set up to allow users to ask ‘famous people questions’. Its homepage suggests that it is designed for any kind of ‘famous person’ though seems to be dominated by UK politicians. Looks interesting but can’t help wonder if there are other sites which could facilitate a ‘discussion’ just as well or better.

The dius section of the site seems quite new. In fact my (rather quickly composed) question was the second to be added to the site. I think the idea of practitioners (yuck, did I just use that word?) raising issues directly with Ministers is an interesting one, and hope it takes off, and at very least, he/they answer the questions!

DIUS do seem to be making an effort to use web2.0 tools. I recently came across this sandbox idea of collecting sites from delicious based on tags, in this example, the library2.0 tag. Interesting stuff, but not specific to HE, it will work for any tag and really just creates a nice view of the latest items bookmarked with the tag in question. The code for it is here.

In any case, it is good to see a government department trying out such tools and also releasing the code under the GPL (even 10 Downing street’s flickr stream is under crown copyright, and don’t get me started on OS maps and Royal Mail postcodes). I’m reminded of the Direct.gov team who, when they found out there was a ‘hack the government‘ day to mashup and improve government web services, decided to join in.

DIUS homepage with web2.0 tools
DIUS homepage with web2.0 tools

On the DIUS hompage, just below the fold, they have a smart looking selection of tools, nice to see this stuff here, and so prominent, though the Netvibes link to me just a holding page when I tried it.

Finally, they have set up a blog on the jiscinvolve (WordPress MU) site. At the time of writing it has a few blogs posts which are one line questions, and has a couple of (good) responses. But I can’t help feeling that these sites need something more if they are to work. At the moment they are just there floating in space. How can they integrate these more into the places that HE staff and students inhabit. Perhaps by adding personal touches to the sites would encourage people to take part, for example the blog – a set of questions – is a little dry, it needs an introduction, host, and photos.

To sum up, some good stuff going on here, but need to see if it takes off, it must be difficult for a government department to interact with HE and students, the two are very different but they are trying.  I hope it proves useful, if you’re involved in HE why not take a look and leave a comment?

UPDATE: Posted on