icon icon

Web Scraping and API in Python

icon

Training process

Training needs analysis

If you have specific requirements regarding the training programme, we will carry out a training needs analysis for you. This will guide us on which aspects of the programme should receive greater emphasis, so that the training programme meets your specific needs.

What will you gain?

icon

Stronger HTTP understanding - You will learn to inspect HTTP requests and responses, read status codes and headers, and understand how communication works, so you can troubleshoot issues and build cleaner integrations.

icon

Efficient data extraction - You will learn how to locate elements in HTML and the DOM with CSS selectors and XPath, so you can pull specific data from detail pages, result lists, and paginated websites.

icon

Python scrapers from scratch - You will practice downloading pages with requests and parsing them with BeautifulSoup, so you can build your own scripts that export structured data to CSV and JSON files.

icon

Confident API usage - You will master sending API requests, handling JSON, working with API keys and tokens, and dealing with errors and rate limits, so you can connect external services with confidence.

icon

Combining multiple data sources - You will learn how to collect information from different APIs and websites, then merge it into one consistent dataset that you can use for reporting, analysis, or further automation.

icon

Automation for dynamic pages - You will use Selenium to click through pages, fill out forms, log in, and capture JavaScript-generated content when scraping static HTML alone is not enough for the task.

icon

More reliable scripts - You will learn practical rules for rate limiting, robots.txt awareness, error handling, and reducing the risk of blocks, so your scraping scripts run more reliably and with fewer interruptions.

icon

Business-ready use cases - You will turn the techniques into practical outcomes by preparing solutions for price monitoring, market data aggregation, automated reporting, and data integrations that support business decisions.

Training programme

1. Introduction to HTTP and how the network works

  • client–server model,
  • HTTP methods (GET, POST, PUT, DELETE),
  • headers, response statuses,
  • structure of an HTTP request and response,
  • tools for traffic analysis (DevTools).

2. Basics of web scraping

  • what web scraping is and when to use it,
  • HTML and DOM structure,
  • identification of elements on the page (CSS selectors, XPath),
  • limitations and good practices (robots.txt, rate limiting).

3. Web scraping using requests and BeautifulSoup

  • retrieving HTTP pages (requests),
  • parsing HTML (BeautifulSoup),
  • extracting data from documents,
  • handling pagination and multiple subpages,
  • saving data (CSV, JSON).

4. Communication with API

  • what API is and how it works,
  • types of API (REST),
  • authorization (API key, token),
  • sending requests and receiving responses,
  • working with JSON,
  • error handling and request limits.

5. Practical use of API

  • integration with external services,
  • retrieving and processing data,
  • combining data from different sources,
  • automation of processes using API.

6. Selenium – automation and scraping of dynamic websites

  • introduction to Selenium,
  • browser control,
  • interaction with page elements (clicks, forms),
  • data extraction from dynamic websites (JavaScript),
  • handling login and sessions,
  • headless browser and task automation.

7. Good practices and optimization

  • script stability,
  • avoiding blocks and bot detection,
  • time and performance management,
  • code structuring and reuse.

8. Business applications

  • price and competition monitoring,
  • aggregation of market data,
  • reporting automation,
  • integrations of systems and data sources.

What are the prerequisites for participating in the training?

icon

Basic Python skills - You should be comfortable running scripts and using variables, conditions, loops, and functions, so you can focus on networking, APIs, and data processing during the course.

icon

Terminal and package basics - You should know how to use a terminal, install libraries with pip, and work in a simple coding environment, so you can prepare your exercises and follow the examples smoothly.

icon

Basic HTML understanding - You should recognize common page elements such as headings, links, tables, and forms, so you can better understand document structure and identify the data to extract.

icon

Analytical problem solving - You should be able to break a problem into steps and verify what your script returns, because scraping and API work often require you to investigate the source of errors.