Database Administrators Stack Exchange
Database Administrators Stack Exchange

Questions tagged [bigtable]

Bigtable is a wide-column NoSQL data storage system designed and built by Google.

Bigtable is a compressed, high performance, and proprietary data storage system built on Google File System, Chubby Lock Service, SSTable (log-structured storage like LevelDB) and a few other Google technologies. It is not distributed outside Google, although Google offers access to it as Google Cloud Bigtable, part of Google Cloud Platform.

9 questions
12
votes
1 answer

What are the performance implications of polymodels versus normal Bigtable models in Google App Engine?

What produces the best performance in regular Google App Engine use, a polymodel or a normal "Bigtable" model? The polymodel, effectively, creates a column in the parent table called "class" which provides the inheritance tracking. Whereas a normal…
Brian Ballsun-Stanton
  • 4,683
  • 2
  • 27
  • 36
9
votes
1 answer

Why can't we perform joins in a distributed database like Bigtable?

From this answer to "What database does Google use?": Bigtable is not a relational database. It does not support joins nor does it support rich SQL-like queries. Is this obvious? I do not understand why.
Lazer
  • 3,251
  • 15
  • 38
  • 53
3
votes
1 answer

Replacing an aggregation system of statistics

We currently have a database under MySQL, storing aggregated statistics in different tables (recent hours, hours, days, months). The tables are updated by workers running at different rates depending on the freshness required for the data. Then…
Dysosmus
  • 131
  • 1
2
votes
1 answer

Can you use BigQuery to run on top of Bigtable

I need to run BigQuery on top of Bigtable live, not as an export. I have found the information stating it was in beta but only as an export function. I would like to run BigQuery against Bigtable data without exporting. Has anyone done this or is it…
cmydata
  • 21
  • 2
0
votes
0 answers

Are there bigtable uses that are anti-patterns? Especially, will doing many writes of the same data (same key) be an issue?

I've been using Cassandra and have run into various problems with Tombstones¹. These would cause detrimental issues when I would later run a query. For example, if I overwrite the same few rows over and over again, even though I still have 5 valid…
Alexis Wilke
  • 111
  • 7
0
votes
0 answers

Running BigQuery on BigTable vs loading from Google Storage

I have a medium size dataset (a few TBs), that is changing over time that requires BigQuery analysis. The first attempt to do analysis was composed by: dumping the DB into a JSON moving it into a Google Storage bucket loading the dataset with…
M4rk
  • 111
  • 2
0
votes
0 answers

Choosing right database for storing bank transactions

I am starting a new project within GCP and I am trying to choose a right tool for storing bank transactions: I don't need transactions, these will be basically write-only, no updates I don't need joins and relations I need grouping and sums Ideally…
Vojtěch
  • 89
  • 3
0
votes
2 answers

How to speed up an insertion from a huge table with postgres?

I have 5 tables in my database with respectively a size of 70Gb, 500Mb, 400 Mb, 110Mb and 20 Mb. I want to create a new table that contains all columns of all tables, so I tried 2 queries, the first one is : select into new_table as select .. from…
Islacine
  • 27
  • 5
-1
votes
1 answer

suggestion needed for big data development

I am trying to find out what is state of the art with database, python, and big data. My starting point began with a SQL server, and multiprocessing pandas, and dask. Imagine I need to maintain a database with more than 1 billion rows, and I need to…
thinker
  • 121
  • 4