Meet the Headitors

Late last year I was fortunate to be invited by Al Jazeera for the Media in Context hackathon in Doha, Qatar. In an effort to predict the future of news, Al Jazeera put together data scientists, developers, journalists, designers, civil activists and mentors from all over the world in one room for three days to solve innovation challenges. I gravitated towards Building Smarter Headlines challenge where I met Sofia (journalist) and Kartik (developer) – and boom!!! The Headitor was born.

headitors(Chris Orwa, Sofia Lotto, Kartik Mandaville)

The headitor is (was intended to be) a dynamic headlines generation platform that suggests different headlines for different demographics and social media platforms using machine learning algorithms. In essence, generating 7 billion headlines for each individual on planet earth. We started by gathering headline from the New York Times and USA Today via their respective APIs .

Second, we filtered for tweeted articles and got the number of retweets on each headline via the Twitter API. At this point we were then be able to suggest replacement words that are likely to attract more readership. Third, we built a functionality that summarised an article written by a journalist, then suggest words/phrases suitable for headlines on Twitter, Facebook and LinkedIn by a subjectivity-sentiment score. Voila! We had a Minimal Viable Product.

headAnother cool idea as shown in the image above was re-writing of headlines by age, since certain words never existed before their inventions alternative words replace them with older words.

Here’s the presentation:

Although we didn’t win a price a number of news outlets covered the ‘product’.

Sadly Headitors Inc had to close shop after 4 long days in business. To our esteemed customers, you can still find the code on GitHub – Happy Hacking!

What Happened in 1995?

I started a mini project a while ago on collecting Kenyan currency note serial numbers for retrospective study. The exercise was simple, get a note and record the value of the note, serial number and date of printing. First slice at the data brought up an oddity, in 1995 currency printing was done twice in a year, on January 1, 1995 and  July 1, 1995.  This phenomenon led to diving in more on the data and further patterns came up:

  • Production of 10 shilling note halted and replaced with a coin.
  • The 1000 shilling note was introduced.
  • A new series of serial numbers was deployed.

KenyaP24c-10Shillings-1994-donatedorus_b

Notes:
Double production means CBK wants more money in the market.
Introduction of 1000 shilling note and dropping of 10 shilling note indicate inflation.

In order to understand why this happened I chose to follow other similar data points. These are;

  • Forex Bureaux
  • Exchange Rate
  • Interest Rate
  • Inflation
  • GDP

Forex Bureau
In 1995, the Central Bank of Kenya (CBK) licensed the first batch of 19 forex bureau. Prior to 1995, only the CBK transacted in the  FX market (anyone traveling out of the country obtained dollars from the CBK and subsequently deposit them on re-entry to the country).  This signified an economic expansion by creation of additional jobs as well as contribution to the Exchequer.  Herein are the Bureau De Change licensed in 1995:

forex

 

Forex Exchange Rate

“SAP- induced reforms in the first quarter of 1994 instituted a free-floating exchange rate policy in Kenya, with the value of the Kenyan shilling thereafter determined by its supply and demand in international money market. Prior to the reform, the Kenyan government followed a fixed-exchange regime in which the shilling was pegged to the US dollar at a specific rate, subject to alteration only to rectify substantial distortion.”

Interest Rate 

From the graph should below it can be seen that in 1995 negative interest rate which means you get charge and interest when you deposit cash. This is one of the government policies to encourage people to spend money instead of keeping it away for future use. Instead of earning interest on money saved you loose a certain portion of it.

interest

 

GDP

From 1996 the nominal GDP in Kenya experience an exponential growth.

 

GDP

 Conclusion

The Kenyan government commenced a series of sectorial reforms to usher the country into a new financial age. It is my observation that if anyone needs to undertake an economic/financial analysis he/she needs to go as far back as 1995 – that’s our modern day economic ground zero.

Discourse on Kenya Rebasing to Middle Income Status

Featuring
Leonida Mutuku: Research Manager
Patrick Costello: Data Scientist
Christine Mahihu: Research Assistant
Chris Orwa : Executive Producer ;)
14860074544_42e0fb1bfa_zImage Courtesy of Jesse McKee

SET

(iHub Coffee bar, everyone having a cup of their favourite stimulant)

Chris Orwa: Hey guys, check out this article

Christine Mahihu: Kenya will become middle-income state by September.

 Leonida Mutuku: Patrick, any thoughts on this rebasing?

 Patrick Costello: Sounds like a useful exercise although the move to middle income status will definitely mean changes for things like donor funding. Should create benefits such as attractiveness for foreign investment – you could measure this via relative impact on Kenyan government funding rate.Might take a few years to show up. Does change in status affect borrowing rates? Is there a pickup in rate of FDI with change in status? How much donor funding, subsidies etc actually gets lost in the move.

Christine Mahihu: I do know, Kenya lost access to funds when it moved from underdeveloped country to a developing country in terms of climate change project financing. We also lost access to the European carbon market. May not be directly related, but this move stalled major projects countrywide and forced involved organization to raise funds elsewhere, something that could also happen in other sectors if this change in status actually happens.

 I like Patrick’s highlight on the benefits of rebasing. The earlier analysis seems to concentrate more on the negative. I also wonder what path other countries took when they were faced with the same situation. I agree, this is a great opportunity to compare the positive and negative effects. With regards to the carbon market, the idea was to provide funding to developing and underdeveloped countries so that they can take a more ‘green’ approach in their path towards development. Unfortunately, China was the biggest beneficiary (and did not necessarily reduce emissions) causing the EU to rethink funding to developing countries, where both China and Kenya fall. The whole thing is big and complicated but no use getting into that.

 Patrick Costello: Hmmm, it’s interesting to note from that bit the rebasing only just squeezes Kenya over the middle income status at $1,036. I think you could make a pretty good argument to hold-off on cancelling donor aid/preferential treatment if cutting that aid pushed you immediately back to down poor country status, and that aid should only be stopped if removing the aid did not push a country back to poor income status.

 Also remember that the downside of loss of donor funding/world bank loans due to increased economic development should be offset by lower risk premiums on loans, hence cheaper loans from regular sources over the longer term. E.g as Kenya is seen to be more stable and a better investment hopefully Kenyan government credit rating goes up and government funding goes down which will lower Kenyan borrowing rates (as most debt will price at a spread to government debt). Will be interesting to see if this happening in practice, but should be easy to track. In fact it would be interesting to look at credit rating vs development status.

 In relation to carbon market access – I don’t think this is development related, rather an EU decision affecting eligibility of all CDM (Clean Development Mechanism) projects across the board post 2012. Not sure why they made it, there’s a pretty opaque description about wanting to develop some new migration scheme.

How Not to Innovate

You have a novelty idea that would take the world by storm, good! What do you do next? Get a couple of friends and begin to hacking it away. Soon you have a minimum-viable-product and ready to meet the market. Everything at this stage is experimental, the budget low and everyone who has anything to do with the product is a brand ambassador. Then lady luck strikes, product sells, gains market share and the idea transforms to a company. What’s wrong with the aforementioned scenario? Read it once again. Steps taken to create a legal entity, formalize operations, attract investors and have an acceptable product actually inhibit innovation in a company. Chaos theory demands that innovation thrives at the edge of chaos’, an environment that balances the need for order and the pull of change. No one captures the essence of this balance better than Michael Crichton in his book ‘The Lost World’.

We imagine the edge of chaos as a place where there is enough innovation to keep a living system vibrant, and enough stability to keep it from collapsing into anarchy. It is a zone of conflict and upheaval, where the old and new are constantly at war. Finding the balance point must be a delicate matter-if a living system drifts too close, it risks falling over into incoherence and dissolution; but if the system moves too far away from the edge, it becomes rigid, frozen, and totalitarian. Both conditions lead to extinction. By implication, extinction is the inevitable result of too much change or too little.

Some people may refer to it as organized chaos or chaordic (chaos & order) arrangement, but the bottom line is, innovation requires a degree of randomness to flourish. A factor pushed away by introduction of business processes. Think about tech startup companies in beta phase, normally the companies would recruit volunteer beta testers and also solicit for reviews on product amiability as well as performance. Once revenue starts trickling in they no longer need your opinion. Let’s get started with Klout, when Twitter was picking up momentum in 2010 Klout managed to convince everyone that their interactions amounted to a measurable level of influence. They had everyone’s attention, and what did they do with it? Ignore pertinent questions about their algorithms as they were busy working out an acquisition plan. But that’s not what irks me, several years later I would wake up to completely different dashboards quiet regularly, no notification, no invitation for beta testing, no nothing. Our mutual relation had ended; we were no longer partners in this innovation business. This goes for Facebook too, at the formative stage Facebook had a link next to the ads where anyone would sign up to be a beta tester and suggest new features. An ecosystem that crowdsourced ideas but someone thought it was bad for business as it ate up advertisement space. When I get asked which common mistake most startups make, the most obvious one is they focus on product and fail to innovate on their processes.

Tracking An Entrepreneur: Kenya’s 80’s Millionaire

I love rhumba music, that’s how I stumbled on the website Congo in Kenya,  an impressive genealogy of Congolese musicians working in East Africa in the 70’s and 80’s.  But what captured my imagination was a photo (shown below) of a membership ticket to The Starlight Club in Milimani, by then a heaven for performing Congolese artists. To put it into context, in the 80’s, Milimani was (and still is) an upmarket neighborhood in Nairobi. Add that to the explosive nature of rhumba music in the 80’s and you have a millionaire in the form of the club owner.

So, I began the quest to know who owned the club and the story begins on 8th September 1980 when James Njau Njenga gets admitted to the Kenyan bar in Gazette Notice No. 3180. The young lad soon hit the ground running by using his registered P.O Box number 44450 (same as on the membership card) on 22 May 1981 to get a license for the bus number KMA 953 to ply Western, Nyanza, Nairobi, Coast, Eastern and North-Eastern and Rift-Valley provinces. It is in the same period that he must have gained ownership of the club after the demise of JM Kariuki who was the director of the establishment.

With the establishment under his arms, the young entrepreneur now had Kenya’s bright and influential mingling in the club including James Ngugi later changing his name to Ngugi wa Thiongo and Senior Barrack Obama. Practicing as an advocate of the high court (employment number P.105/1150/80 ), Mr. Njenga took home 150,000 Kshs, factoring inflation and the two known businesses he was definitely in the league of millionaires. In 1994, the curtains came down on the infamous Starlight Club as the Kenyan government through the Attorney General acquired the piece of land (LR 209/1069)  it sat and that’s where the magnificent Integrity House sits today.

However, the story doesn’t end there. Mr. Njau through a newly acquired post office number P.O Box 62097 setup a private practice at Shankardass House along Moi Avenue from where he fanned out his new “business empire”. First up is Thuo Investment Company Ltd, an entity dealing in buying and selling of land and estates. Next up is Afcon Ltd, a company providing resources, knowledge and expertise to the development of safe water to the people in East Africa. Mumbi Properties ltd LR 209/4283

Unfortunately, Mr. Njenga died on 17th September 1997 after  his vehicle registration number KAE 125P was hit by a truck belonging to Pelican Haulage Contractors Limited (registration number KXN041) along Langata Road. His wife, Phyllis Wangui Njau was awarded a compensation worth 5,031,760 Kshs on a ruling rendered on 14th December 2012  by Justice Manyanja and each of his 3 children got Kshs 2 million. Since then, two companies using his old P.O Box number 44450 have been registered namely; Bright Homes in Milimani and Pearl Banquets Ltd operating on Argwings Kodhek. Hope it is the entrepreneurial bug passed from father to child.

References

Afcon. (2013). Contact Us. Retrieved November 27, 2013, from Afcon Ltd: http://www.afconltd.co.ke/contact-us/

Johnston, A. (2013, November 19). Congo in Kenya. Retrieved November 27, 2013, from Congo in Kenya: http://muzikifan.com/shika.html

Kenya Law. (2012). In the Matter of the Estate of James Njenga Njau. Nairobi: Kenya Law.

Kenya, R. o. (1981). The Kenya Gazette. Nairobi: Republic of Kenya.

Republic of Kenya. (1980). The Kenya Gazette. Nairobi: Republic of Kenya.

Thuo Investment. (2013). Contact Us. Retrieved November 27, 2013, from Thuo Investment: http://www.thuoinvestments.com/index.php?option=com_contact&view=contact&id=1&Itemid=54

Rare Clouds Over Nairobi and Cairo

On the 8th day of January, 2014 the year of our lord, the Gods of Nairobi and Cairo  bequeathed the two cities with a similar and rare morning skyline.

                                                                               NAIROBI

What I saw when I stepped out of the crib this morning. I was late, but had to stop to capture God's artistry

A photo posted by Conrad Gray (@gmoneyizme) on

Good morning from #nairobi #kenya

A photo posted by Samir Dave (@samdave69) on

Good morning. #sunrise #nairobi #Kenya #Africa #nature

A photo posted by Sir Fredrick (@pseudosanity) on

Good morning Nairobi, beautiful sunrise

A photo posted by Gituma Nturibi (@onegitts) on

                                                                              CAIRO

#Clouds, Now #Cairo

A photo posted by Hanii Ghazal (@longlivesamra) on

Capturing Tweets with R

Its been a labor of love trying to capture tweets for analysis via R, for starters R is an open source statistical computing software and perhaps best in its league. Leveraging its prowess in analyzing textual data and in particular tweets is a thing to marvel at, so let’s get on how to setup R to capture and analyze tweets. First, register an app with Twitter, it’s pretty simple. Go to https://dev.twitter.com/ and create a new application, of course you have to have Twitter account, if that’s the case you should be seeing the page below.

Create An Application

Good, you are doing well. Key in the details, you can guess the website as well as the Callback URL, they aren’t important at the moment. Next, after successful creation of the app click on My Application and choose you app. This where it starts to get interesting, you should be able to view details of your app with the important facts being below the title OAuth SettingsFor my app, these are the details.

Access level Read-only

Consumer key 8mzRs9PySHKmTcvXBcy5w
Consumer secret ZKNBKniG4ADfyk3tHCWQsj0wowapFpXhqoj8O4OnQQ
Request token URL https://api.twitter.com/oauth/request_token
Authorize URL https://api.twitter.com/oauth/authorize
Access token URL https://api.twitter.com/oauth/access_token
Callback URL None
Sign in with Twitter No

Now you got the feeling you are doing something, time to fire up R. The twitteR package contains the functions for querying Twitter servers, you proceed on by downloading the package (install.packages(“twitteR”)) or if you already have it installed simply call it via require(twitteR) function. Some versions of the package miss the dependency packages ROauth and RCurl, if that’s the case you’ll have to download the packages independently (install.packages(ROauth) and install.packages(RCurl)). Proceed with providing the authentication details provided in your app, keep in my the ROauth package has the authentication function and if it isn’t working the process would halt. Use the code shown below:

code

That’s pretty much standard code, note the consumer key and consumer secret key should be the one provided for you app (I’m using mine). The next part is the tricky section and gave me some headache. After providing the credentials via the OAuthFactory$new() function a system handshake has to be initiated between you app and Twitter server. A handshake in computing is a prior communication between two systems that sets the rules of communication, in this case it is implemented by digital certificate (SSL certificates) sent from Twitter server acknowledging the app and setting type of information to be communicated. The straight and faster way to go round this is to first download the certificate with the R code:

download.file(url="http://curl.haxx.se/ca/cacert.pem", destfile="cacert.pem")

Then proceed with the code below to initiate the handshake.

twitCred$handshake(cainfo="cacert.pem")

Success at this stage should request for a pin in a textual format, copy paste the URL.

To enable the connection, please direct your web browser to:

https://api.twitter.com/oauth/authorize?oauth_token=kxzyNUke8nBprcClN4BTipXqgWKKn27Xf7We1qPJZE

When complete, record the PIN given to you and provide it here:

This is what I got, punch back the PIN on the prompt in the above reply.

Ingokho PIN

We are almost there, next is to register the credentials, the function returns TRUE when all is well.

registerTwitterOAuth(twitCred)

At this point I felt home is just a stone throw away only to be hit with the an error while trying to use the search Twitter function.

[1] "SSL certificate problem, verify that the CA cert is OK. Details:\nerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"
Error in twInterfaceObj$doAPICall(cmd, params, "GET", ...) : 
  Error: SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

If this happened to you too do not worry I have the antidote. Set the SSL globally using the code below.

library(RCurl) 

# Set SSL certs globally
options(RCurlOptions = list(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl")))

Horay!!! we’ve done it. Now we have the power to search, collect and analyze tweets. How about we do something interesting? Let’s download tweets from a trending topic and graph who are the highest contributors to the topic. Download and install the ggplot2 and plyr packages, using the searchTwitter() function in twitteR package I captured 1000 tweets from the trending topic #TvYa13Million and graphed it. Here is the code that did all the magic: [PS: I didn’t exit my R sessions, this code is a continuation of the above]

require(ggplot2)
library(plyr)

TvTweets = searchTwitter("#TvYa13Million",n=1000)
users <- ldply(TvTweets,function(x) return(x$screenName))
ggplot(users,aes(x=V1))+geom_histogram()+theme(axis.text.x = element_text(angle = 45, hjust = 1))+ylab("Count of tweets using #TvYa13Million")+xlab("Twitter handle")

And there you go:TvYa13Million

Thank you for reading, you now have the power to capture tweets and everything pertaining to analysis.

Adios!!!