This module provides a comprehensive, practice-oriented foundation in modern data engineering and data analytics, equipping students with the technical skills required to process, manage, and extract value from diverse data sources. Participants will learn to use Python and its core libraries (NumPy, Pandas, Matplotlib, and SciPy) for data transformation and analysis, while developing a solid understanding of Big Data system architectures, distributed cluster computing, and cloud-based data services.
The module also covers essential topics such as data conversion and standardization, data preparation techniques for statistical modelling and machine learning, and effective data visualization. In addition, students will be introduced to emerging technologies such as Blockchain and its underlying security principles, enabling them to operate confidently in both established and evolving data-driven environments.
By the end of the module, students will be able to design and implement end-to-end data workflows—from data ingestion and cleansing to analysis and the communication of insights—using industry-standard tools and best practices.
To successfully participate in this module, students are expected to have a basic understanding of data processing, including familiarity with databases, common data formats, and both traditional and advanced data analysis techniques. Proficiency in the Python programming language is important, as it serves as the primary tool for implementing the concepts and practical exercises covered throughout the course. In addition, a basic understanding of cloud architectures is recommended, as many of the technologies and workflows addressed in the module are designed to operate in distributed or cloud-based environments.
The aim of Big Data Analysis is to introduce students to the tools and technologies required to manage the vast volumes of data generated in today's digital world. This will be achieved through a combination of data management architecture (the Big Data Pipeline), the use of specialized Big Data processing technologies, and practical programming in Python. By the end of the course, students will be able to process large datasets and manipulate data to generate statistics, metrics, and visualizations, ultimately extracting valuable business insights from data.
- Introduction to Big Data
1.1 What is Big Data?
1.2 Types of Data
1.3 The Data Value Chain
1.4 Data Management and Data Governance
1.5 The Big Data Pipeline
1.6 The Emerging Role of the Data Engineer - The Big Data Pipeline
2.1 Architecture and Methodology
2.2 Information Systems Environments
2.3 General Data Processing Patterns
2.4 The Reference Architecture for Information Systems - Data Ingestion
3.1 Types of Data Sources
3.2 Data Ingestion Approaches
3.3 Batch Processing and Online Processing
3.4 Synchronous and Asynchronous Data Ingestion - Structured Data Repositories
4.1 Fundamentals of Database Systems
4.2 Data Warehouses
4.3 Data Models
4.4 Types of Queries
4.5 Data Warehouse (DWH) Use Cases - Semi-Structured and Unstructured Data Repositories
5.1 The Data Lake
5.2 Data Lake Use Cases
5.3 The Lakehouse Concept
5.4 NoSQL Databases
5.5 NoSQL Database Use Cases - Data Security and Blockchain
6.1 Fundamentals of Cryptography and Data Security
6.2 Blockchain Architecture
6.3 Blockchain Use Cases - Data Analytics
7.1 Traditional Analytics and Business Objectives
7.2 Analytics and Data Processing Patterns
7.3 Decision-Making
7.4 Data Visualization
7.5 Advanced Analytics
7.6 Machine Learning Techniques
7.7 Transformers and the Attention Mechanism - Distributed Data Processing
8.1 Characteristics of Processing Clusters
8.2 Hadoop
8.3 Spark
8.4 Stream Processing - Distributed Data Storage
9.1 Data Storage Architecture
9.2 Storage Clusters
9.3 Partitioning and Replication
9.4 The HDFS Architecture
9.5 Major Distributed Storage Solutions - Cloud and Big Data
10.1 Understanding Cloud Computing
10.2 Cloud Architecture and Service Models
10.3 Leading Cloud Data Services and Platforms - Information Systems Maintainability
11.1 Protection Strategies
11.2 Extensibility, Simplicity, and Operability
This module consists of two teaching sessions per week. Each session is divided into two parts: the first is primarily instructor-led, during which new concepts and theoretical foundations are introduced; the second is dedicated to practical exercises, allowing students to reinforce and consolidate the knowledge acquired. Student progress is assessed regularly through participation in individual and group activities, the submission of assignments and take-home exercises, and other assessment activities.
A significant proportion of the practical activities and exercises will be carried out using standard or academic cloud platforms, enabling students to become familiar with this type of environment.
During the final sessions of the module, students will complete a group project in teams of four to demonstrate their understanding and application of the concepts covered throughout the course.
The teaching methodology therefore combines independent learning, collaborative classroom activities, and continuous formative assessment, ensuring alignment between learning activities, the assessment system, assessment criteria, and the student workload associated with the allocated ECTS credits.
To evaluate whether students have achieved the intended learning outcomes of the module, a range of assessment activities is used throughout the semester, typically on a weekly basis.
The table below shows the weighting of each assessment activity towards the final grade:
CONTINUOUS EVALUATION SYSTEM
Evaluation type | Weight | Content | Activity type | AIAS Level |
Attendance and participation | 25% | All topics | Moderately important | 1 |
Activities | 35% | Around 5 o 6 activities individual or groupal | Highly important | 4 |
Mid-term exam | 10% | Covered tòpics to date | Moderately important | 1 |
Final exam | 30% | All topics | Highly important | 1 |
Students who do not pass the ordinary assessment period will be eligible to take an Extraordinary Assessment, consisting of a resit examination. To be eligible for this assessment, students must have submitted all outstanding assignments and practical exercises. The final grade for the Extraordinary Assessment will be calculated using the same assessment weightings as those applied during the continuous assessment process, with a maximum attainable grade of 6.0.
Students who do not attend the resit examination will receive a final grade of NP (Not Presented) for the Extraordinary Assessment.
A minimum grade of 3.0 out of 10 must be obtained in both the final examination and the resit examination in order to pass the module.
The following aspects will be assessed:
- The correct application of the concepts presented in class when solving the proposed exercises.
- The quality of the conclusions drawn from the activities and the accurate interpretation of the results obtained.
- The clarity, organization, and structure of the presentation of procedures and solutions.
- The appropriate and authorized use of Artificial Intelligence (AI) in accordance with the module guidelines.
Marin, I., Shukla, A., & VK, S. (2019). Big Data Analysis with Python. Packt Publishing.
Martin Kleppmann, (2019), Designing Data-Intensive Applications [O’Reilly]
---