Python on Firas Sadiyah

Python Exception Handling

Sat, 24 Feb 2024 12:00:00 +0000

In genomic data analysis, we often use a pipeline function to process data stored in a dataframe by calling several mini-functions. Each mini-function may modify the dataframe by adding a new column with new values and then filter out the rows that do not meet certain criteria. However, this may result in an empty dataframe if none of the rows satisfy the filters and can lead to errors or unexpected results when the pipeline function tries to perform more operations on the empty dataframe. To avoid this situation, we can use two strategies. First, we can check if the DataFrame is non-empty before applying any logic in each mini-function. Second, we can make the pipeline function fail graciously if it receives an empty DataFrame from any of the mini-functions by using a custom exception and a try-except block. Let’s take a look.

Directing Python Output in Terminal

Sun, 18 Feb 2024 12:00:00 +0000

When writing Python scripts, it’s common to want to display information messages in the user terminal and possibly save them to a file like output.log. Therefore, it’s essential to understand the levels of logging in Python, the types of outputs in Unix-like systems, and how to direct the appropriate type of logging to the right output.

Understanding Output Streams in Unix

In Unix systems, there are two primary streams for output: standard output (stdout) and standard error (stderr). These streams serve different purposes:

Running PyEnsembl on CentOS HPC

Sun, 11 Feb 2024 12:00:00 +0000

Today, I am installing a Python package on one of the high-performance computing (HPC) systems that I work with. This CentOS 7 setup resembles a fortress, with its stringent security protocols, understandable due to hosting clinical data, but making it quite a challenge to install essential Python packages from public repositories.

The package I’m attempting to install is PyEnsembl, a handy tool for accessing Ensembl’s genetic information.

PyEnsembl Setup

I started by creating a new conda environment:

Debugging Python codebases using PyCharm and VSCode

Sun, 04 Feb 2024 12:00:00 +0000

Debugging, an essential process for identifying and rectifying errors in a computer program, is particularly crucial for computational biologists dealing with complex codebases that often involve intricate mathematical models, data analysis, and simulations. Merely reading the code may not suffice to grasp the logic and functionality of the project. To gain a deeper understanding, you may need to run the code, examine the variables, and observe the outputs. Towards this end, the integrated debuggers in PyCharm and VSCode prove invaluable.

Managing Python virtual environments

Sun, 28 Jan 2024 12:00:00 +0000

When it comes to Python development on macOS, I rely on a combination of two tools that have served me exceptionally well over the past few years: Pyenv and Poetry. Pyenv provides an elegant solution for managing different Python versions on my system, while Poetry simplifies dependency management and the creation of virtual environments for my projects. In this article, I will guide you through the process of setting up Pyenv to install a specific Python version and then using Poetry to create a virtual environment for your project.

Using reticulate in RStudio with pyenv

Wed, 28 Apr 2021 00:00:00 +0000

When developing in Python, it is generally a good practice not to rely on the Python version that ships with the operating system (OS). This is to ensure that the system version of Python remains relatively ‘clean’ for the OS processes. In addition, by installing a custom version(s) of Python, we open many possibilities. For one, it gives us control over which specific version(s) to use in our projects, and two, by using a virtual environment manager, we ensure that each project has access to its own tailored list of packages. One way of achieving this is by using pyenv.

Managing Python virtual environments

Tue, 01 Aug 2017 12:00:00 +0000

Installation

 1# macOS
 2brew install python3 python-pip
 3
 4# Ubuntu
 5sudo apt install python3 python-pip
 6
 7# Update pip to the latest version
 8pip install --upgrade pip setuptools wheel
 9pip3 install --upgrade pip setuptools wheel
10
11# Install virtualenvwrapper
12pip3 install virtualenv virtualenvwrapper

Create projects directory

1mkdir ~/Projects

Configure .profile (Bash) or .zprofile (ZSH):

1# needed for virtualenvwrapper
2export WORKON_HOME=$HOME/.virtualenvs
3export PROJECT_HOME=$HOME/Projects
4export VIRTUALENVWRAPPER_PYTHON=/bi/home/USER/.linuxbrew/bin/python3
5export VIRTUALENVWRAPPER_VIRTUALENV=/bi/home/USER/.linuxbrew/bin/virtualenv
6export PIP_REQUIRE_VIRTUALENV=true
7source /bi/home/USER/.linuxbrew/bin/virtualenvwrapper.sh

Reload profile

1source ~/.profile
2source ~/.zprofile

Make a new project

The command will create a python virtual environment by running mkvirtualenv.
Then it will creates a project directory and will turn to that directory.

1# using the default python version
2mkproject projectName
3
4# using specific python version
5mkproject -p `which python` projectName

To delete a project

1rmvirtualenv projectName

Bonus

If you are using ZSH shell with prezto (recommended), you can change prezto prompt to indicate the virtual environment currently in use. You need to edit prompt_sorin_setup as follow¹ :