Python

In genomic data analysis, we often use a pipeline function to process data stored in a dataframe by calling several mini-functions. Each mini-function may modify the dataframe by adding a new column with new values and then filter out the rows that do not meet certain criteria. However, this may result in an empty dataframe if none of the rows satisfy the filters and can lead to errors or unexpected results when the pipeline function tries to perform more operations on the empty dataframe. To avoid this situation, we can use two strategies. First, we can check if the DataFrame is non-empty before applying any logic in each mini-function. Second, we can make the pipeline function fail graciously if it receives an empty DataFrame from any of the mini-functions by using a custom exception and a try-except block. Let’s take a look. ...

When writing Python scripts, it’s common to want to display information messages in the user terminal and possibly save them to a file like output.log. Therefore, it’s essential to understand the levels of logging in Python, the types of outputs in Unix-like systems, and how to direct the appropriate type of logging to the right output. Understanding Output Streams in Unix In Unix systems, there are two primary streams for output: standard output (stdout) and standard error (stderr). These streams serve different purposes: ...

Today, I am installing a Python package on one of the high-performance computing (HPC) systems that I work with. This CentOS 7 setup resembles a fortress, with its stringent security protocols, understandable due to hosting clinical data, but making it quite a challenge to install essential Python packages from public repositories. The package I’m attempting to install is PyEnsembl, a handy tool for accessing Ensembl’s genetic information. PyEnsembl Setup I started by creating a new conda environment: ...

Debugging, an essential process for identifying and rectifying errors in a computer program, is particularly crucial for computational biologists dealing with complex codebases that often involve intricate mathematical models, data analysis, and simulations. Merely reading the code may not suffice to grasp the logic and functionality of the project. To gain a deeper understanding, you may need to run the code, examine the variables, and observe the outputs. Towards this end, the integrated debuggers in PyCharm and VSCode prove invaluable. ...

When it comes to Python development on macOS, I rely on a combination of two tools that have served me exceptionally well over the past few years: Pyenv and Poetry. Pyenv provides an elegant solution for managing different Python versions on my system, while Poetry simplifies dependency management and the creation of virtual environments for my projects. In this article, I will guide you through the process of setting up Pyenv to install a specific Python version and then using Poetry to create a virtual environment for your project. ...

When developing in Python, it is generally a good practice not to rely on the Python version that ships with the operating system (OS). This is to ensure that the system version of Python remains relatively ‘clean’ for the OS processes. In addition, by installing a custom version(s) of Python, we open many possibilities. For one, it gives us control over which specific version(s) to use in our projects, and two, by using a virtual environment manager, we ensure that each project has access to its own tailored list of packages. One way of achieving this is by using pyenv. ...

Installation 1# macOS 2brew install python3 python-pip 3 4# Ubuntu 5sudo apt install python3 python-pip 6 7# Update pip to the latest version 8pip install --upgrade pip setuptools wheel 9pip3 install --upgrade pip setuptools wheel 10 11# Install virtualenvwrapper 12pip3 install virtualenv virtualenvwrapper Create projects directory 1mkdir ~/Projects Configure .profile (Bash) or .zprofile (ZSH): 1# needed for virtualenvwrapper 2export WORKON_HOME=$HOME/.virtualenvs 3export PROJECT_HOME=$HOME/Projects 4export VIRTUALENVWRAPPER_PYTHON=/bi/home/USER/.linuxbrew/bin/python3 5export VIRTUALENVWRAPPER_VIRTUALENV=/bi/home/USER/.linuxbrew/bin/virtualenv 6export PIP_REQUIRE_VIRTUALENV=true 7source /bi/home/USER/.linuxbrew/bin/virtualenvwrapper.sh Reload profile 1source ~/.profile 2source ~/.zprofile Make a new project The command will create a python virtual environment by running mkvirtualenv. Then it will creates a project directory and will turn to that directory. 1# using the default python version 2mkproject projectName 3 4# using specific python version 5mkproject -p `which python` projectName To delete a project 1rmvirtualenv projectName Bonus If you are using ZSH shell with prezto (recommended), you can change prezto prompt to indicate the virtual environment currently in use. You need to edit prompt_sorin_setup as follow1 : ...