Bioinformatics 1

Using Python Notebooks

mcorey — Tue, 01 Aug 2023 10:51:26 +0000

Using Python Notebooks mcorey Tue, 01/08/2023 - 11:51

Those of you that already know how to code using Python will likely be familiar with "Notebooks" already, but for those of you who are not these are "executable" ways of working with code, meaning that you can write some code and then run it inside the notebook so that you can immediately see what the code does (e.g. does it work!).

In the first week's lecture I will show you how to use a notebook so don't worry if you aren't comfortable with doing so now.

IF you would like to get started the easiest way to start is to download the files below and then upload them to the University's "Noteable" system (which you can access here). You will first need to login using your university UUN and then you will be taken to the Noteable launch page:

Step 1 - click the link it will take you here:-

Step 2 - the default is fine so click "Start" and this will load to a screen like this:-

Step 3 - now click on the "upload" box in the top-right and upload the python notebook file you would like to use (you can download the ones for this mini-course below).

Step 4 - I've uploaded the first file from this mini-course "Python_Basics_1_part_1.ipynb", click on this to run the notebook:-

Step 5 - You should now see the notebook running on the screen:

That should be you up and running. We will go through this in the first lecture as well.

License

About us

mcorey — Mon, 31 Jul 2023 15:28:12 +0000

About us mcorey Mon, 31/07/2023 - 16:28

Where are you all from?

We thought it would be nice to know where you are all from, we suspect we have a big international group on the course. You can do this by putting a pin in the Padlet below. You can attach your name to it if you like but there's no need to if you prefer not to.

About Your Instructors

Ian Simpson (e-mail)
I am Professor of Biomedical Informatics in the School of Informatics and Director of the UKRI Centre for Doctoral Training in Biomedical Artificial Intelligence. My methodological research interests include:

database systems
algorithms
high-performance computation
software engineering
agent-based modelling
computational statistics
applied machine learning
natural language processing
knowledge representation

My research group are largely focussed on studying various aspects of neurodevelopmental diseases from both genomic and clinical perspectives and use a wide range of data types and analytical techniques to learn more about disease aetiology with one of our key objectives being to close the genotype-phenotype gap. Much of our current work is centred on autism spectrum disorders & intellectual disabilities. In these projects we are using knowledge representation implemented as feature rich network (graph) structures. Using both computational statistical and graph learning approaches we can examine the features of these graphs to better understand patients.

I am an avid Road, Cyclocross and MTB cyclist riding with Edinburgh Road Club for whom I also coach as a GO-Ride, Regional & National cluster coach. I have been a STEM Ambassador for many years working with local Schools in particular and enjoy taking part in public engagement activities. My family is obsessed with board-gaming so if you want some tips about good games do please ask!

Barry Ryan (e-mail)

Hi, I’m a PhD student in the School of Informatics. I’m from Cork in Ireland and my research interests include:

• Multi-modal Data Integration
• Graphical Neural Networks
• Genetics of Parkinson’s Disease

I enjoy Sport and the outdoors and in particular watching Rugby. I took Bioinformatics 1 last year so if you have any questions feel free to reach out at any stage.

License

Course Contacts

flittlet — Tue, 18 Jul 2023 14:11:46 +0000

Course Contacts flittlet Tue, 18/07/2023 - 15:11

Teaching Team

Course Organiser - Prof. Ian Simpson (e-mail)
Course Secretary - Yesica Yesica Marco Azorin (e-mail)
Teaching Support - Barry Ryan (e-mail)

License

Resource List

flittlet — Tue, 18 Jul 2023 14:11:43 +0000

Resource List flittlet Tue, 18/07/2023 - 15:11

Bioinformatics 1 Resource List

You can access the reading list for this course by selecting the link above. In order to view some resources on the list, you must be logged in with your EASE account.

For more information on getting the most out of your courses Resource List, have a look at this video.

License

Week 10 - Gene Expression Analysis

flittlet — Tue, 18 Jul 2023 14:11:43 +0000

Week 10 - Gene Expression Analysis flittlet Tue, 18/07/2023 - 15:11

This week in the lecture I will be presenting differential gene expression analysis using experimental data derived from microarrays and RNA-sequencing. I will show how the outcomes from these analyses can be interpreted with the help of some of the downstream methods we have been learning about in the second half of the course.

Whilst this is not part of the assessed content for the course it is such a widely used data modality that we wanted to make sure that you had some experience and awareness of it including where data can be sourced and the primary approaches, methods, and tools used.

It's also an opportunity to also ask more general questions about research topics, careers, and ways to further develop your interest in Bioinformatics in the future.

We hope you've enjoyed the course and that it has sparked greater interest in the potential to apply computational approaches to important biological and biomedical problems.

Lecture 10 - Gene Expression Analysis

The lecture slides for Week 10 will be available here.

The video of the lecture will be available from the GitHub video area here.

Following this course you may well want to learn more, one excellent second semester course available in Edinburgh is Next Generation Genomics.

There is an excellent relevant Chapter in our course textbook, Pevsner, for this and next week - Chapter 9 - "Analysis of Next-Generation Sequence Data"

Reading Lists & Resources

Each week we will have an accompanying reading list with some articles & web-sites for self study to support the course. You can find the course "Resource List" - here. We will continue to curate the list throughout the course especially if things pop up in the lectures and practicals that we want to add a reference or link to so do please check back in on the list from time to time.

We have generally tried to identify resources as "Essential", "Recommended" or "Further Reading" in an attempt to help you prioritise your reading during the course.

Finally a very important time to draw your attention to what you can consider the "core text" for the course, which is the excellent "Bioinformatics & Functional Genomics" Third Edition by Jonathan Pevsner. You will be pleased to know that this text-book is available free online as part of the University's subscription portfolio. You can find it right at the top of the resource list. If you have any problems accessing or using any of the above please do drop us a comment in the Discussion forum and we will try to get things resolved as soon as possible.

License

Week 6 - Working with Biological Databases

flittlet — Tue, 18 Jul 2023 14:11:42 +0000

Week 6 - Working with Biological Databases flittlet Tue, 18/07/2023 - 15:11

This week we are going to be practicing working with biological databases in the computing lab session. For the lecture this week I will be walking through and explaining a series of Jupyter Notebooks that programatically access data carrying on from the introduction last week and including some solution code. These notebooks are well annotated and there for you to use, modify and play with whenever you want to. This will hopefullly inspire you in Python coding. As you will see this week, it can empower you to perform large and complex studies taking advanatge of the vast array of data resources and services available from all over the world. We are keeping reading low this week as I know that a lot of people will be working hard on the coursework.

The notebooks are, as ever, available on our GitHub site - https://github.com/tisimpson/bioinformatics1/tree/main/labs/notebooks

Here is some extra information on working with the python pandas library that I mentioned last week and which is used in the latest notebooks in case you find this useful.

Data Handling Using Pandas

There is a very popular python library that some of you may already heard of called 'Pandas' - Python Data Analysis Library that we use in all of the notebooks this week. It is a package that has created specific data strcutures and functions that can act upon them to do a vast array of things that are common asks in data analysis. It has grown to such an extent that there are many books written about it and the website itself is a very high quality resource explaining how to use it with examples and reference material. If you are going to learn data science and use Python then Pandas is the library that you are going to want to learn how to use.

Pandas - https://pandas.pydata.org/
Pandas documentation - https://pandas.pydata.org/docs/
and this 10 minute quick-start is very useful too - https://pandas.pydata.org/docs/user_guide/10min.html#min

O'reilly have published a book "Python for Data Analysis" that covers Pandas, Numpy and iPython (effectively notebooks) that you might find very useful to have as a reference book in the future.

Lecture 6 - Working with Biological Databases

The lecture slides for Week 6 - "Exploring Biological Databases" are available here. This week I will be continuing demonstrations of database use from the web live as well as going through some solutions to the lab challenges in the lecture so there are only a few slides. This will also mean that people could choose to catch up next week if they are pressed for time with the coursework deadline just after the lecture this Friday.

The video of the lecture is available from the GitHub video area here.

Reading Lists & Resources

We have generally tried to identify resources as "Essential", "Recommended" or "Further Reading" in an attempt to help you prioritise your reading during the course.

This week you should browse BFG for examples of some of these databases in use, especially material about biological databases in Chapter 2, but you don't need to read the whole chapter. Other useful things you might like to browse are some fo the useful guides provided by some of the resources above such as:-

NCBI Training Tutorials - https://www.ncbi.nlm.nih.gov/guide/training-tutorials/ (very good)
PubMed User Guide - https://pubmed.ncbi.nlm.nih.gov/help/ (very good)
Bioportal Help - https://www.bioontology.org/wiki/BioPortal_Help
Biogrid Help - https://wiki.thebiogrid.org
Reactome User Guide - https://reactome.org/userguide (very good)

It will take more time than we have in the course for you to become comfortable with all of the resrouces. The aim here is to introduce you to them and give you some experience of using them (which we will do next week) so that in the future you will know where to start when looking for the different kinds of data.

License

Week 5 - Exploring Biological Databases

flittlet — Tue, 18 Jul 2023 14:11:42 +0000

Week 5 - Exploring Biological Databases flittlet Tue, 18/07/2023 - 15:11

For the next two weeks we are going to be learning about a range of databases that are available to interrogate and gather data about many of the biological systems and processes that we study. This week we will focus on becoming familiar with what’s available, what it can be used for and, crucially, how to recover that data and begin working with it ourselves. Taking that forward we will be working through some case studies next week in some of the common application areas.

Introduction

Exploring Biological Databases

This week we are going to be learning about a selection of databases that are available to interrogate and gather data about many of the biological systems and processes that we study. This week we will focus on becoming familiar with what’s available, what it can be used for and, crucially, how to recover that data and begin working with it ourselves. Taking that forward we will be working through some case studies next week in some of the common application areas.

Data Handling

There are many ways to obtain and subsequently work with biological data, but typically we can break these down into those that use tools and those that involve coding or scripting. This mirrors the discussion we had in week 1 about the different ways in which researchers carry out their work, both are equally valid, but they both also have consequences.

Tools-based access

Broadly speaking, tool based access is more accessible and often specifically designed to allow the user to achieve common tasks and requests without too steep a learning curve. Also, behind the scenes considerable effort will often have been put in to make sure the tool performs exactly as designed minimising the risk of imprecise specification in requests and in return formats. It is also easy to share what was done with other, in fact many such tools offer the ability to save and share searches which can facilitate data sharing and open research practices. On the downside not all sources of data may provide such tools, requiring fairly low-level direct access to data often in the form of large and frequently complex flat-files. For those sources that do provide tools, they may be quite restrictive in what parts of the data you can access, how you can build your queries and what format it can be returned in.

Programmatic access

It is fair to say that most experienced Bioinformaticians will use a programmatic method to access and process data once they get beyond the preliminary exploratory phase. This can be anything from low-level UNIX commands such as 'curl' and 'sftp', shell-scripts that collate many commands together in simple programs, though to scripting languages such as Perl, Python and R that commonly have packages for accessing specific data sources. All of the major sources of biological data expose services for programmatic access either through direct FTP like downloads of raw data or through database or API level access to services. Increasingly these data sources are employing advanced modern methods for exposing services rather than designing bespoke systems in a specific programming language. This means that a Bioinformatician can commonly access and interrogate the data source using a programming method of their choosing. Clearly, the versatility of accessing data through a programmatic route is often greater, but it does come with risks. It is not uncommon for custom code written by Bioinformaticians to not perform as intended. This could be due to misunderstanding the data structures and services offered by the data source or poor code.

Databases for Genes - NCBI & Ensembl

Gene Databases (Part One)

Gene Databases (Part Two)

Literature Searching - PubMed

In the video we refer to a few useful pages that contain helpful information for using PubMed.

Searching PubMed

Structuring Data for Biology - BioPortal & EBI-OLS

In this video we introduce ontologies as structures that enable us to annotate and analyse biological data in a consistent and comparable manner and encode prior knowledge about the biological systems we are studying.

Protein-Protein Interaction Data - Intact & BioGrid

In this video we introduce two of the largest protein-protein interaction databases BioGrid & Intact (EBI). We discuss some of the factors to consider when querying and interpreting data from including the importance of recognising and evaluating the confidence levels associated with individual interactions. These databases are a nice example of how reporting standardisation (through the HUPO PSI-MI3.0 data definition) and incorporation of bespoke ontologies for protein interactions (MI Ontology) has allowed for strong provenance and the development of statistical approaches to evaluate (and infer) interactions.

Biological Pathways - KEGG & Reactome

In this video we introduce KEGG and Reactome (EBI), two pathway database resources that can help us to gain biological insight into the data we are analysing. They essentially allow us to view our data from the perspective of the biological pathways and systems that effect higher level properties such as cell-cycling, neurotransmitter signalling, glycolysis etc.. These are by nature hierarchical meaning that these can be viewed across large biological scales from very precise molecular mechanistic reactions up to whole tissue or organ systems. These resources map across species as well which allows researchers to take results from model organisms (for example) and project them into analogous systems in other species such as the human.

Jassal B, Matthews L, Viteri G, Gong C, Lorente P, Fabregat A, Sidiropoulos K, Cook J, Gillespie M, Haw R, Loney F, May B, Milacic M, Rothfels K, Sevilla C, Shamovsky V, Shorser S, Varusai T, Weiser J, Wu G, Stein L, Hermjakob H, D'Eustachio P. The reactome pathway knowledgebase. Nucleic Acids Res. 2020 Jan 8;48(D1):D498-D503. doi: 10.1093/nar/gkz1031. PMID: 31691815; PMCID: PMC7145712.

Gene Expression Data - NCBI-GEO & ArrayExpress

In this video we introduce GEO (NCBI) & ArrayExpress (EBI) two databases that allow researchers to query and download vast quantities of gene expression data for analysis. Both repositories enforce very strict standards on submissions to ensure that data is of suitable quality and accompanied by the essential meta-data required to allow for reproducible and properly designed analysis. These are additional examples of initiatives that have established and/or adopted community standards of practice to maintain quality and engender confidence from their scientific user base.

Lecture 5 - Exploring Biological Databases

The lecture slides for Week 5 - "Exploring Biological Databases" are available here. This week I will be demonstrating use of the databases we use from the web live in the lecture so there are only a few slides.

The video of the lecture is available from the GitHub video area here.

Reading Lists & Resources

We have generally tried to identify resources as "Essential", "Recommended" or "Further Reading" in an attempt to help you prioritise your reading during the course.

NCBI Training Tutorials - https://www.ncbi.nlm.nih.gov/guide/training-tutorials/ (very good)
PubMed User Guide - https://pubmed.ncbi.nlm.nih.gov/help/ (very good)
Bioportal Help - https://www.bioontology.org/wiki/BioPortal_Help
Biogrid Help - https://wiki.thebiogrid.org
Reactome User Guide - https://reactome.org/userguide (very good)

License

Week 7 - Ontologies & Functional Enrichment Analysis

flittlet — Tue, 18 Jul 2023 14:11:42 +0000

Week 7 - Ontologies & Functional Enrichment Analysis flittlet Tue, 18/07/2023 - 15:11

This week we will be learning about ontologies. These structured resources allow us to used semantically meaningful restricted vocabularies to annotate biological data and enable rigorous computational analysis. Ontologies are key to imposing structure on the largely unstructured data we deal with as Bioinformaticians. By using standardised concepts to describe things we enable key essential features of quantitative research and inter-operability.

We will also be learning about ways that we can use ontologies and annotation in two different quantitative approaches to gain insight from the lists of genes and proteins that are often produced by laboratory experiments.

Lecture 7 - Ontologies & Functional Enrichment Analysis

The lecture slides for Week 7 - "Ontologies & Functional Enrichment Analysis" are available here.

The video of the lecture will be available from the GitHub video area here.

Reading Lists & Resources

We have generally tried to identify resources as "Essential", "Recommended" or "Further Reading" in an attempt to help you prioritise your reading during the course.

This week we have two journal articles for you to read, which are on the resource list, but here also for convenience:

- Aleksander, Suzi A., Seth Carbon, J. Michael Cherry, Harold J. Drabkin, Marc Feuermann, Pascale Gaudet, et al. 2023. “The Gene Ontology Knowledgebase in 2023” edited by A. Baryshnikova. Genetics (Austin) 224(1). doi: 10.1093/genetics/iyad031.

- Subramanian, Aravind, Pablo Tamayo, Vamsi K. Mootha, Sayan Mukherjee, Benjamin L. Ebert, Michael A. Gillette, Amanda Paulovich, Scott L. Pomeroy, Todd R. Golub, Eric S. Lander, and Jill P. Mesirov. 2005. “Gene Set Enrichment Analysis: A Knowledge-Based Approach for Interpreting Genome-Wide Expression Profiles.” Proceedings of the National Academy of Sciences - PNAS 102(43):15545–50. doi: 10.1073/pnas.0506580102.

License

Week 8 - Biological Network Analysis

flittlet — Tue, 18 Jul 2023 14:11:42 +0000

Week 8 - Biological Network Analysis flittlet Tue, 18/07/2023 - 15:11

This week we will be looking at networks and how they can be used to help us study biological systems. We will be using protein-protein interaction, gene expression, and literature networks as exemplars. We will also learn some of the fundamentals of how networks are created and analysed. Additional optional reading for the week is available in the course resource list.

Lecture 8 - Biological Network Analysis

The lecture slides for Week 8 - Biological Network Analysis" are available here.

The video of the lecture is available from the GitHub video area here.

Reading Lists & Resources

We have generally tried to identify resources as "Essential", "Recommended" or "Further Reading" in an attempt to help you prioritise your reading during the course.

- Xiaoke Ma, Lin Gao. Biological network analysis: insights into structure and functions. Briefings in Functional Genomics, Volume 11, Issue 6, November 2012, Pages 434–442 https://doi.org/10.1093/bfgp/els045 - Analysis of Biological Networks

- Junker, Björn H., and Falk. Schreiber. Hoboken, N.J: Wiley-Interscience, 2008 Chapters 1,2 and 9 (protein-protein interactions, esp. sections 9.3 and 9.4)

These are available via the course resource list.

License

Week 9 - Introduction to Next Generation Sequencing

flittlet — Tue, 18 Jul 2023 14:11:42 +0000

Week 9 - Introduction to Next Generation Sequencing flittlet Tue, 18/07/2023 - 15:11

Lecture 9 - Introduction to Next Generation Sequencing

The lecture slides for Week 9 are available here.

The video of the lecture is available from the GitHub video area here.

Introduction

There have been several revolutions in DNA sequencing technologies over the last 20 years that have transformed biological research, redefining not just how things are done, but the very nature of what experiments are possible. In particular the ready availability of high fidelity, high throughput DNA seqeuncing at low cost has both democratised research and allowed us to look into biological systems in unprecedented detail. There are entire courses dedicated to the analysis of data from sequencing technologies and we will not be able to go into the same detail as those. However, we feel that it is essential that any course teaching bioinformatics today needs to at least familiarise students with the technologies, their potential, application, and the some of the downstream analysis possible. Following this course you may well want to learn more, one excellent second semester course available in Edinburgh is Next Generation Genomics.

There is an excellent relevant Chapter in our course textbook, Pevsner, for this and next week - Chapter 9 - "Analysis of Next-Generation Sequence Data"

Reading Lists & Resources

We have generally tried to identify resources as "Essential", "Recommended" or "Further Reading" in an attempt to help you prioritise your reading during the course.

License