October 16, 2017
In recent years cloud storage such as Dropbox or Drive is gaining popularity across different industries principally for its affordability given the excellent performance offered. Within this domain, also cloud databases are now challenging well-established solutions deployed on site.
Regarding big-data analytics, cloud databases are now starting to play a crucial role. According to a recent survey by O’Reilly Media, around 40% of data sources are composed of non-RDBMS sources such as Hadoop, NoSQL, in-memory and search databases. 20% are columnar/MPP analytic databases and a fast-growing 10% are cloud databases such as Amazon Redshift and Google BigQuery. The survey also points out that a shrinking 30% of data analytics is still performed against traditional relational database management systems.
One of the reasons why companies are considering of moving to a fully managed, scalable, low-cost cloud databases is because they generally have multiple tools deployed across different departments. Frequently, this leads to having a dedicated IT teams busy in building and maintaining pipelines for each tool at the cost of agility and resources. On the contrary, a cloud solution would minimise IT effort freeing budgets for the analytics side.
Here at SSIX, our partner 3rdPLACE employed Google BigQuery, a cloud database which provides a performing infrastructure for storing and analysing terabytes of data in seconds without thinking about servers, indexing, distribution strategies or maintenance. Specifically, we are capturing live feeds into multiple BigQuery tables which are then made available for processing and algorithms training.