Big Data and Data Engineering
BASE Properties; CAP Theorem; OLTP vs OLAP
C-CAT
BASE Properties
What is BASE?
BASE is the consistency model used by NoSQL databases (opposite of ACID):
| Letter | Property | Definition |
|---|---|---|
| BA | Basically Available | System guarantees availability (not every request will get the latest data, but a response) |
| S | Soft State | The state of the system may change over time (even without new inputs — due to eventual consistency) |
| E | Eventually Consistent | System will become consistent over time (not immediately; all replicas eventually agree) |
Example of Eventual Consistency:
User posts a photo on Instagram (write to primary replica)
Immediately visible on their phone
After 1-2 seconds: visible globally (replicas catch up)
Eventually consistent — not immediately consistent
CAP Theorem
What is CAP Theorem?
CAP Theorem (Brewer's Theorem, 2000): A distributed system can guarantee at most 2 out of 3 properties simultaneously:
| Property | Description |
|---|---|
| C — Consistency | Every read receives the most recent write or an error |
| A — Availability | Every request receives a response (not necessarily latest) |
| P — Partition Tolerance | System continues operating despite network partitions |
C (Consistency)
/\
/ \
/ \
/ \
CA ------+--------+ CP
/ \
/ \
/ AP \
+________________+
A (Availability) P (Partition Tolerance)
Network partitions ALWAYS happen in distributed systems → you must choose between C and A.
Database Classification by CAP
| Type | Prioritizes | Examples |
|---|---|---|
| CP | Consistency + Partition Tolerance (sacrifice Availability) | HBase, MongoDB, Redis |
| AP | Availability + Partition Tolerance (sacrifice Consistency) | Cassandra, DynamoDB, CouchDB |
| CA | Consistency + Availability (only on single node — no partition) | Traditional RDBMS |
OLTP vs OLAP
OLTP — Online Transaction Processing
| Feature | OLTP |
|---|---|
| Purpose | Day-to-day operations; transactions |
| Data | Current, operational data |
| Queries | Simple CRUD; INSERT, UPDATE, DELETE |
| Response time | Milliseconds |
| Users | Thousands of concurrent users (clerks, customers) |
| Data size | GB |
| Normalized | Highly normalized (3NF) |
Examples: Banking transactions, e-commerce orders, inventory management
OLAP — Online Analytical Processing
| Feature | OLAP |
|---|---|
| Purpose | Analysis, reporting, decision support |
| Data | Historical, aggregated data |
| Queries | Complex aggregations, multi-dimensional analysis |
| Response time | Seconds to minutes |
| Users | Few analysts and managers |
| Data size | TB to PB |
| Denormalized | Star schema, snowflake schema |
Examples: Sales trend analysis, customer behavior analysis, financial reporting
OLTP vs OLAP Comparison
| Feature | OLTP | OLAP |
|---|---|---|
| What | Record transactions | Analyze data |
| Query type | Simple (INSERT, simple SELECT) | Complex (GROUP BY, JOIN across years) |
| Users | Many concurrent (1000s) | Few (10s) |
| Data | Current (days) | Historical (years) |
| Schema | Normalized | Denormalized (star/snowflake) |
| Optimization | Fast transactions | Fast reads at scale |
| Database | MySQL, PostgreSQL | Snowflake, BigQuery, Redshift |
Continue learning
Related notes
Definition of AI; Need of AI
Artificial Intelligence
Introduction to Data Engineering; Big Data — The 5 V's; Types of Data
Big Data and Data Engineering
Introduction to C Programming; C Program Structure; Data Types and Variables
C Programming
What Is a Computer?; Machine Cycle: Fetch–Decode–Execute; CPU Organization
Computer Architecture
Put this topic into timed practice
Open mock tests when you want full-exam pacing, or keep drilling in practice mode.