We use cookies,
see info/options.
If you're OK with
cookies click below
or continue browsing.
Proceed
Cookie Information: 
Cookies are text files containing small amounts of data which are downloaded to your computer, or other device, when you visit a website.

Cookies are useful for carrying out various tasks, including improving your experience on our website. Some cookies are also necessary for the technical operation of our website. Cookies do not harm your computer.

For information about your cookie options including turning them off, click here.

To carry on with cookies running, click proceed or click the X to close this window and continue browsing. You can review your cookie options at any time by clicking on the Cookies link at the foot of each page
Proceed

Discover patterns
in data

DataMine Lab is a data mining and business intelligence company We provide data and software consulting to the advertising industry

Recent Posts

  • YCSB run against HBase 0.92 on Amazon Elastic MapReduce
  • BigData events
  • R Analytics in the Cloud
  • BigData London events (and a ticket giveaway)
  • Calculating unique visitors in Hadoop and Hive

BigData events
May 4, 2012 by Radek Maciaszek,  2 comments

We observe an explosion of BigData events. While half a year ago London hosted maybe one interesting meetup a month nowadays there is rarely a week without few of them. Supply is keeping up with demand.

There is an increasing number of monthly meetups: BigData London, HUG UK, Data Science London, London R, Cassandra London, Neo4j London, London MongoDB User Group, Oracle BigData, Data Visualisation London, Big Data Debate, DeNormalised London, LonData, CloudComputing.

Upcoming conferences that are worth mentioning:

We just had a London BigData week that was full of meetings and hackatons dedicated to Hadoop, Visualisations and NoSQL. In case you missed the last Big Data week you are for a treat – simply like us on Facebook to have a chance of winning one ticket (worth £495) for 3 days of SkillsMatter NoSQL tutorials.

There are as well few online places where every data scientist can improve or challenge their skills:

If you know of anything interesting coming up in London, let us know in the comments.

R Analytics in the Cloud
November 21, 2011 by Radek Maciaszek,  no comments

Last week I was invited to Big Data London to talk about “R Analytics in the Cloud”. As a case study, I presented the ageing project I’ve been working on as part of my Masters studies at Birkbeck, University of London. Ageing is one of the fundamental mysteries in biology and many scientists are already studying this process. I am excited to be part of the research group led by Eugene Schuster at UCL Institute of Healthy Ageing. This project has also given me the chance to use some of my Hadoop experience in the academic field.

Bioinformatics is the science of applying information technology to biology in order to understand the latter. There are numerous ways in which computers can aid biologists. In this particular project, we have been using microarrays to find the connection between different genes. The use of microarray technologies has enabled us to detect changes to gene expression across the genome in thousands of experiments with hundreds of species. However, interpreting the changes identified in these experiments has been hampered by a lack of knowledge of the gene function. Even in highly studied genomes, approximately 50-60% of genes will be assigned functions, yet less than 30% will be annotated with a highly specific function. Little of the annotation will have been observed in experiments conducted with the species of interest, as most gene function annotation is based on annotations assigned to orthologous genes taken from experiments done with other species, such as yeast and mammalian cell culture.

We are interested in building a better understanding of gene function in the worm C. elegans by harnessing the large quantity of experimental microarray data in the public database. Currently, we have a database of over fifty curated experiments. With this, we attempt to assign putative functions to genes based on the expression profile across experiments in the public repositories. My role in this project is to help expand the number of curated experiments in the database and study the functions of approximately 1000 genes known to be regulated in long-lived worms, to try to understand the functions of these genes, e.g. by showing experimental evidence of a role in nutrient sensing, innate immunity or stress response.

Here are the slides from the presentation. Refer to slides 10 and 11 to see how to migrate your R application to the cloud in just 3 lines of code:

Oh, and did I mention how cool our lab is? Have a look at the following ad, which was made at UCL  just a couple of metres from my desk.

Full disclosure: DataMine Lab is in no way affiliated with Birkbeck or UCL and the above project is part of my individual bioinformatics studies.