Python Project Walkthrough with L2D
We thought it would be good to take a look at Python, the most widely used programming language. We are working with Learn To Discover (L2D), a training provider specialising in Python skills, to produce a short series of blog posts showcasing some of the strengths and weaknesses of Python. This is the first of those posts, covering setting up your machine and installing the necessary tools to work with their example code.
Introduction to L2D
L2D offer a fully-supported online training course in Python programming, data science, network science, classical machine learning and artificial intelligence (AI) tailored for health, disease and bioscience. They provide real-time, taught training programs with instructor support. They’re also a supporter of the AIBIO-UK network, and we’ll be offering a number of funded places on their Complete Course – watch this space!
Python strengths and weaknesses
Python is the most-widely used programming language, and it’s only becoming more dominant, but all languages have their drawbacks. We discussed some of the best things about Python, and some of the things that can cause frustration.
Strengths
- Simplicity
Python code is generally fairly easy to understand, and this is baked into the syntax. Code has to be structured neatly for it to run. Errors are meaningful and well-described. This makes it relatively easy to learn, and quick to write. Python is a “high level” language that abstracts out a lot of the details. This means you don’t have to worry about things like memory management, and it’s possible to write succinctly. This comes at a cost of slower-running code, but many Python libraries offset this by interfacing with higher-performance languages like C/C++. - Adoption
Python has a huge ecosystem. There are hundreds of thousands of open-source packages, covering a vast array of tasks. Tutorials are easy to find, questions get answered on StackOverflow, and LLMs can offer decent help thanks to plenty of training data. In many fields Python is the de facto standard, so understanding it makes it easier to replicate work or join a new organisation.
Weaknesses
- Performance
For many projects it takes much longer to write code than it does to run it, so performance isn’t always an issue. However, when handling large datasets or when speed is critical, Python doesn’t have great support for parallelisation, and developing high-performance code can require interfacing with other languages. - Fragmentation
The downside of the huge ecosystem is that there isn’t always a common standard. In many common tasks – like installing packages and managing dependencies and environments, making visualisations, developing deep learning algorithms, or working with data frames – there are several widely-used alternatives. For visualisation, there are dozens libraries. A developer starting a new project must evaluate and choose between these options, and understanding other’s code becomes more challenging.
Project walkthrough
This will be a four part series showcasing one of the main strengths of Python: it has an enormous and thriving ecosystem of libraries and tools, particularly in machine learning and AI. We’ll start by installing the tools we need and creating a clean environment, then we’ll explore a dataset and build some models, first using classical machine learning then trying out a deep learning approach. This will be covered in four articles:
The L2D team have prepared the code, and we’ve published it in a GitHub repository. If you’re familiar with setting up a project and managing environments and want to get straight to the code, you can find the repository here. The code is written in Jupyter notebooks, so you can read the annotations and follow along by executing the code yourself. For an introduction to Jupyter notebooks click here.
If you’d like to learn about Git and Github, you can read our explainer article on Version control with Git.
Otherwise, read on for setup instructions!
Setup
Apart from Python itself, there are a few tools you’ll need to use to clone the repository and install its dependencies. In particular, we’ll be using uv. These are the steps we’ll be following:
- install
uv - clone the repository
- create a virtual environment and install the dependencies
- open the notebook in the virtual environment
Why uv?
If you’re using Mac or Linux, you should already have some version of Python installed. Python works as a programming language for the user, but it also handles critical operating system functions under the hood. Projects in Python have dependencies – sets of packages that implement useful functionality, which could be anything from compressing and decompressing files or sending and receiving network requests, to building visualisations, deep learning models, or user interfaces. These dependencies have their own dependencies, and pretty soon you can end up with hundreds of packages. Problems arise when packages (or their dependencies, or their dependencies’ dependencies…) have incompatible requirements – they need different versions of the same package. Updating a package can break another, and this can interfere with or completely disable a project, or even your operating system, in ways that are very difficult to debug and fix. As you create more projects and continue to work on them, some sort of breaking change between dependencies tends to an inevitability.
Because of this risk, it’s important to keep your projects separate from your main, system Python. This involves several related tasks:
- installing and managing versions of the Python interpreter
- creating environments to insulate the system and projects from each other
- installing and managing dependencies
uv does all of these. If you’re already familiar with all of these concepts and prefer to use other tools then you don’t have to install uv – you could use conda, or pyenv, virtualenv and pip (which comes with Python). We’re using uv here as it’s the simplest and quickest way we’ve found to perform all the above tasks.
Install uv
You can find installation instructions for all systems here.
Clone the repository
If you don’t already have Git installed, follow these instructions.
Once you have Git installed and working, navigate to the directory where you keep your projects and run:
git clone https://github.com/ScryptIQ-ai/AIBio-demo
This should create a new directory called AIBio-demo in the location you run the command from, and download the contents of the repository into it. You should now have the following files:
AIBio-demo
├── 1_data_explore.ipynb
├── 2_classical_ml.ipynb
├── 3_neural_nets.ipynb
├── assets
│ └── fna_pic.png
├── data
│ └── breast_cancer.csv
├── pyproject.toml
├── README.md
└── uv.lock
Along with some hidden files.
Create an environment and install the dependencies
Navigate into the repository directory (AIBio-demo) and run:
uv sync
This small command does a bunch of useful things. The repository contains files, pyproject.toml and uv.lock, that specify the project’s dependencies. uv reads these files, creates a virtual environment if necessary, and installs the dependencies into the environment. See the uv docs for more information.
Open the notebook in the virtual environment
Now we need to launch the notebook using the Python environment that we created and loaded with our dependencies. The simplest way to do this is:
uv run --with jupyter jupyter lab
This launches a Jupyter server and automatically opens a browser window at the right address. You should now be ready to start running code!