CTD Resource Ingest Guide¶
Source Information¶
InfoRes ID: infores:ctd
Description: CTD is a robust, publicly available database that aims to advance understanding about how environmental exposures affect human health. It provides knowledge, manually curated from the literature, about chemicals and their relationship to other biological entities: chemical to gene/protein interactions plus chemical to disease and gene to disease relationships. These data are integrated with functional and pathway data to aid in the development of hypotheses about the mechanisms underlying environmentally influenced diseases. It also generates novel inferences by further analyzing the knowledge they curate/create - based on statistically significant connections with intermediate concept (e.g. Chemical X associated with Disease Y based on shared associations with a common set of genes).
Citations: - Davis AP, Wiegers TC, Sciaky D, Barkalow F, Strong M, Wyatt B, Wiegers J, McMorran R, Abrar S, Mattingly CJ. Comparative Toxicogenomics Database's 20th anniversary: update 2025. Nucleic Acids Res. 2024 Oct 10. PMID: 39385618
Data Access Locations: - CTD Bulk Downloads - http://ctdbase.org/downloads/ (this page includes file sizes and simple data dictionaries for each download) - CTD Catalog - https://ctdbase.org/reports/ (a simple list of files, reports the number of rows in each file)
Data Provision Mechanisms: file_download
Data Formats: tsv, csv, obo
Data Versioning and Releases: No consistent cadence for releases, but on average there are 1-2 releases each month. Versioning is based on the month and year of the release Releases page / change log: https://ctdbase.org/about/changes/ Latest status page: https://ctdbase.org/about/dataStatus.go
Additional Notes: ['CTD has placed a CAPTCHA (ALTCHA) in front of ctdbase.org, which breaks programmatic access to the data status page (https://ctdbase.org/about/dataStatus.go) that reports the current release. The ingest determines the source version by scraping file modification dates from https://ctdbase.org/reports/, which is not currently behind the CAPTCHA. If CTD extends the CAPTCHA to that path, the version detection in the ingest code will need to be revisited.']
Ingest Information¶
Ingest Categories: primary_knowledge_provider
Utility: CTD is a rich source of manually curated chemical associations to other biological entities which are an important type of edge for Translator query and reasoning use cases, including treatment predictions, chemical-gene regulation predictions, and pathfinder queries. It is one of the few sources that focus on non-drug chemicals, e.g. environmental stressors, and how these are related to diseases, biological processes, and genes.
Scope: Covers a diverse range of Chemical association knowledge types provided across several files provided by CTD, based on many knowledge curation/creation approaches.
Relevant Files¶
| File Name | Location | Description |
|---|---|---|
| CTD_chemicals_diseases.tsv.gz | http://ctdbase.org/downloads/ | Manually curated and computationally inferred associations between chemicals and diseases |
| CTD_exposure_events.tsv.gz | http://ctdbase.org/downloads/ | Descriptions of statistical studies of how exposure to chemicals affects a particular population, with some records providing outcomes |
| CTD_chem_gene_ixns.tsv.gz | http://ctdbase.org/downloads/ | Descriptions of curated, detailed causal associations describing how chemicals affect genes/gene products. |
| CTD_chem_go_enriched.tsv.gz | http://ctdbase.org/downloads/ | CTD calculates which GO terms are statistically enriched among the genes/proteins that interact with each chemical or its descendants. |
| CTD_chem_pathways_enriched.tsv.gz | http://ctdbase.org/downloads/ | Inferred associations between chemicals and biological pathways - computationally derived enrichments based on curated chemical–gene and gene–pathway data in CTD. |
| CTD_pheno_term_ixns.tsv.gz | http://ctdbase.org/downloads/ | Curated, literature-based associations between chemicals and "phenotypes". Note that CTD represents these phenotypes as GO terms (biological processes, molecular functions, cellular components) rather than as disease-like phenotypic features. |
| CTD_genes_diseases.tsv.gz | http://ctdbase.org/downloads/ | Manually curated and computationally inferred associations between genes and diseases (not yet ingested - see future considerations) |
| CTD_curated_genes_diseases.tsv.gz | http://ctdbase.org/downloads/ | The manually curated subset of the gene-disease associations above (not yet ingested - see future considerations) |
Included Content¶
| File Name | Included Records | Fields Used |
|---|---|---|
| CTD_chemicals_diseases.tsv.gz | Only the manually curated rows - those carrying a DirectEvidence code of "therapeutic" or "marker/mechanism". Currently, these are a small fraction of the file: of ~9.83M rows, ~39.8K are "therapeutic" and ~69.7K are "marker/mechanism", with the remaining ~9.72M (98.9%) being CTD's inferred chemical-disease rows, which are excluded. | ChemicalName, ChemicalID, DiseaseName, DiseaseID, DirectEvidence, PubMedIDs |
| CTD_exposure_events.tsv.gz | Only records with an outcomerelationship of "positive correlation" or "negative correlation", and an exposurestressorid, and at least one of diseaseid / phenotypeid. This is a small slice of the file: of ~245.7K rows, ~9.4K are "positive correlation" and ~0.7K are "negative correlation" (the rest are empty, "no correlation", or "prediction/hypothesis"). | exposurestressorid, outcomerelationship, diseaseid, phenotypeid, reference |
| CTD_chem_gene_ixns.tsv.gz | Only records carrying exactly one interaction action, and whose Interaction sentence begins with either the chemical name or the gene symbol (so that a subject/object orientation can be recovered). ~1.83M of ~3.13M rows (58.7%) have a single interaction action. | ChemicalID, GeneID, GeneSymbol, ChemicalName, OrganismID, Interaction, InteractionActions, PubMedIDs |
| CTD_chem_go_enriched.tsv.gz | Records with a corrected p value less than 1e-10 and a highest GO level greater than or equal to 3 (~2.10M of ~6.46M rows). Covers all three GO branches - Biological Process, Molecular Function, and Cellular Component. | ChemicalID, GOTermID, HighestGOLevel, PValue, CorrectedPValue |
| CTD_chem_pathways_enriched.tsv.gz | Records with a corrected p value less than 1e-10 (~0.62M of ~1.70M rows). Pathways are roughly 2/3 Reactome and 1/3 KEGG. | ChemicalID, PathwayID, PValue, CorrectedPValue |
| CTD_pheno_term_ixns.tsv.gz | Only records carrying exactly one interaction action - ~133.6K of ~472.6K rows (28.3%). Every one of those single-action records is an "increases^phenotype", "decreases^phenotype", or "affects^phenotype" action. | chemicalid, phenotypeid, organismid, interactionactions, anatomyterms, pubmedids |
Filtered Content¶
| File Name | Filtered Records | Rationale |
|---|---|---|
| CTD_chemicals_diseases.tsv.gz | Records lacking a DirectEvidence designation | These relationships are only based on computational inference and were found to be not reliably useful enough to include |
| CTD_chem_gene_ixns.tsv.gz | Two rules drop records here. First, any record carrying more than one interaction action (a ' | '-delimited InteractionActions value) is dropped - roughly 1.29M of 3.13M rows (41%). Second, any remaining record whose Interaction sentence begins with neither the chemical name nor the gene symbol is dropped, since the subject/object orientation cannot be recovered from it. |
| CTD_chem_go_enriched.tsv.gz | Records with a corrected p value higher than 1e-10 or a highestGoLevel lower than 3 are dropped. | This removes a large number of weak and overly vague associations. |
| CTD_chem_pathways_enriched.tsv.gz | Records with a corrected p value higher than 1e-10 are dropped. | This removes a large number of weak associations. |
| CTD_pheno_term_ixns.tsv.gz | Records carrying more than one interaction action are dropped - roughly 339K of 472.6K rows (72%). | As with the chemical-gene interactions, these multi-action records describe chains of events across three or more entities that cannot be reduced to one self-contained edge. The scale of the loss here is worth noting: we keep less than a third of this file, so any future work to extract edges from multi-entity interaction records would have a large payoff for this file in particular. |
| CTD_exposure_events.tsv.gz | Records that do not have an outcomerelationship of "positive correlation" or "negative correlation" are dropped - about 96% of the file. Records without an exposurestressorid, or with neither a diseaseid nor a phenotypeid, are also dropped. | The other outcomerelationship values ("no correlation", "prediction/hypothesis", and empty values) do not assert a directional relationship that can be represented with a correlation predicate. Records lacking a stressor identifier, or lacking any outcome identifier, cannot be turned into an edge at all. |
Future Content Considerations¶
edge_content: We should ingest gene-disease association edges from the CTD_genes_diseases.tsv file and/or the curated CTD_curated_genes_diseases.tsv.gz file. Lots of content here. Assess quality and compare to other gene-disease association sources when we do a comprehensive review of content/modeling in this area.
edge_content: Consider ingesting the inferred (non-DirectEvidence) Chemical-Disease associations, which we currently exclude entirely. These are derived by CTD from genes shared between the chemical and the disease, and carry an inference score. Note that these edges were emitted by earlier iterations of this ingest, as biolink:associated_with edges with a knowledge_level of statistical_association and an agent_type of data_analysis_pipeline - we subsequently made a deliberate decision to drop them, because they were not reliably useful enough to justify their volume. If we revisit that decision, they should get a quality cutoff (e.g. on shared gene count, publication count, or inference score) - CTD asserts an inference even when it rests on a single shared gene/publication, which is not really meaningful - and they would still need a predicate weak enough to reflect their indirect basis (e.g. biolink:associated_with). - Relevant files: CTD_chemicals_diseases.tsv.gz
edge_property_content: If we ingest the inferred Chemical-Disease associations described above, consider also capturing CTD's inference score, and the list of genes shared between the chemical and disease that support each inference, as edge properties. - Relevant files: CTD_chemicals_diseases.tsv.gz
edge_content: Revisit the corrected p-value cutoff applied to the enrichment-based Chemical-GO term and Chemical-Pathway associations. We currently require a corrected p-value below 1e-10 (plus a highestGoLevel of at least 3 for GO terms), chosen to cut a large volume of weak and overly vague associations, but this threshold has not been tuned against any downstream measure of usefulness. - Relevant files: CTD_chem_go_enriched.tsv.gz, CTD_chem_pathways_enriched.tsv.gz
edge_content: A few CTD chemical-gene interaction aspects still have no Biolink mapping in the transform, so records carrying them become 'affects' edges with no aspect qualifier (counted as unmapped by the transform). Among the single-action records that reach the transform these are "export" (~420 records), "import" (~185), "reaction" (~5), and "polymerization" (~2). "export"/"import" could plausibly map to a transport aspect with a direction. These are low-volume, but until mapped they silently understate what CTD asserts. - Relevant files: CTD_chem_gene_ixns.tsv.gz
edge_content: Reconsider the wholesale drop of multi-action interaction records, which is the single largest source of content loss in this ingest - about 41% of CTD_chem_gene_ixns and about 72% of CTD_pheno_term_ixns. The rule exists because a chain like "[A increases expression of B] which increases activity of C" cannot become one self-contained edge. But a substantial minority of these records involve only the chemical and the gene, and simply assert several simultaneous effects on that one pair (the most common aspect combinations are expression+reaction, binding+reaction, and phosphorylation+reaction). Roughly 317K of the 1.29M dropped chemical-gene records look like this. Splitting a two-entity multi-action record into one edge per action would recover a large amount of curated content; identifying them reliably (rather than by the current sentence-prefix heuristic) is the hard part. - Relevant files: CTD_chem_gene_ixns.tsv.gz, CTD_pheno_term_ixns.tsv.gz
node_property_content: Molepro ingested chemical properties in its previous ingests - which we will likely bring in at some point. In particular, data from the CTD_chemicals.tsv file - Relevant files: CTD_chemicals.tsv
edge_property_content: The exposures event file provides a lot of provenance/evidence metadata about supporting studies, including characteristics of subject populations, location of the study, and experimental/statistical data values. Consider for inclusion using a StudyResult object. - Relevant files: CTD_exposure_events.tsv.gz
Target Information¶
Edge Types¶
| Subject Categories | Predicate | Object Categories | Knowledge Level | Agent Type | UI Explanation |
|---|---|---|---|---|---|
| biolink:ChemicalEntity | biolink:DiseaseOrPhenotypicFeature | knowledge_assertion | manual_agent | Source CTD data provides assertions about Chemical-Disease relationships that were manually curated from literature by CTD curators. The CTD record used to create this Translator edge has a "T" (therapeutic) as a DirectEvidence code, indicating the chemical to be a "potential" treatment in virtue of its clinical use or scientific study. The broad and imprecise nature of this relationship is best represented with the Biolink predicate 'treats or applied or studied to treat' - which is used when it is not clear whether the chemical shows established efficacy or was merely studied or attempted as a treatment. | |
| biolink:ChemicalEntity | biolink:DiseaseOrPhenotypicFeature | knowledge_assertion | manual_agent | Source CTD data provides assertions about Chemical-Disease relationships that were manually curated from literature by CTD. The CTD record used to create this Translator edge has a 'DirectEvidence' code of "M" (marker/mechanism), indicating that the chemical is either a marker or contributing factor for a condition. This vague/imprecise relationship implies that at minimum there is correlation between the presence of the chemical and condition, and is best represented here using the Biolink 'correlated_with' predicate. | |
| biolink:ChemicalEntity, biolink:ComplexMolecularMixture, biolink:MolecularMixture, biolink:SmallMolecule | biolink:PhenotypicFeature, biolink:Disease | statistical_association | manual_agent | Source CTD data provides correlations between Chemicals and Disease based on a single real-world exposure study, that was curated from the literature by CTD curators. These correlations are separately calculated by authors or curators for each study, not an automated analysis pipeline. The CTD record used to create this Translator edge shows a statistically significant correlation between the studied exposure (usually a chemical) and a measured outcome (usually a disease or phenotype). The direction of the correlation reported by the study is represented using the Biolink 'positively correlated with' or 'negatively correlated with' predicate. Note that the edge is based on results from a single study - and may hold only in the context of the study's design or population. | |
| biolink:ChemicalEntity, biolink:MolecularMixture, biolink:SmallMolecule | biolink:Gene | knowledge_assertion | manual_agent | Source CTD data provides assertions about Chemical-Gene relationships that were manually curated from literature by CTD. The CTD record used to create this Translator edge describes how a chemical affects a particular aspect and/or form of a gene or gene product, and often the direction of the effect (e.g. that it causes increased activity of the gene, or decreased stability of the gene). These causal relationships are represented using the Biolink 'affects' or 'causes' predicate, with qualifiers capturing additional detail about the aspect and direction of the effect. | |
| biolink:Gene | biolink:ChemicalEntity, biolink:MolecularMixture, biolink:SmallMolecule | knowledge_assertion | manual_agent | Source CTD data provides assertions about Gene-Chemical relationships that were manually curated from literature by CTD. The CTD record used to create this Translator edge describes how a gene or gene product affects a particular aspect or form of a chemical entity, and often the direction of the effect (e.g. that it causes increased abundance of a chemical, or decreased stability). These causal relationships are represented using the Biolink 'affects' or 'causes' predicates, with qualifiers capturing additional detail about the aspect and direction of the gene's effect. | |
| biolink:Gene | biolink:ChemicalEntity, biolink:MolecularMixture, biolink:SmallMolecule | knowledge_assertion | manual_agent | Source CTD data provides assertions about Gene-Chemical relationships that were manually curated from literature by CTD. The CTD record used to create this Translator edge describes how a gene or gene product (usually a mutant form) may affect the susceptibility/sensitivity of a cell or organism to a particular chemical exposure. The relationship is represented using the Biolink 'affects sensitivity to' predicate - or one of its directional sub-predicates ('increases/decreases sensitivity to') if a direction is provided. | |
| biolink:ChemicalEntity, biolink:MolecularMixture, biolink:SmallMolecule | biolink:Gene | knowledge_assertion | manual_agent | Source CTD data provides assertions about Chemical-Gene relationships that were manually curated from literature by CTD. The CTD record used to create this Translator edge describes how a chemical may affect the susceptibility/sensitivity of a cell or organism to the actions of a particular gene or gene product (e.g. a signaling molecule like TNF). This relationship is represented using the Biolink 'affects sensitivity to' predicate - or one of its directional sub-predicates ('increases/decreases sensitivity to') if a direction is provided. | |
| biolink:ChemicalEntity, biolink:MolecularMixture, biolink:SmallMolecule | biolink:Gene | knowledge_assertion | manual_agent | Source CTD data provides assertions about Chemical-Gene relationships that were manually curated from literature by CTD. The CTD record used to create this edge reports that a chemical physically binds to a particular gene or gene product, without asserting any resulting effect on the gene. This is represented using the Biolink 'directly physically interacts with' predicate, which is symmetric. CTD reports these as its 'binding' interaction aspect (always as 'affects^binding', with no direction), so unlike the other chemical-gene edges these carry no aspect or direction qualifier. | |
| biolink:ChemicalEntity, biolink:MolecularMixture, biolink:SmallMolecule | biolink:BiologicalProcess, biolink:BiologicalProcessOrActivity, biolink:MolecularActivity | knowledge_assertion | manual_agent | Source CTD data report assertions about chemicals and cellular phenotypes, that were manually curated from literature by CTD. The CTD record used to create this Translator edge reports that a chemical causes a cellular phenotype, which is framed as an affect on a biological process or activity. This relationship is represented using the Biolink 'affects' or 'causes' predicates, with qualifiers capturing additional detail about the direction of the effect, and the anatomical or species context of the effect where provided (e.g. that it causes increased levels of the process or activity in a particular cell type, tissue, or species). | |
| biolink:ChemicalEntity, biolink:MolecularMixture, biolink:SmallMolecule | biolink:BiologicalProcess, biolink:BiologicalProcessOrActivity, biolink:MolecularActivity, biolink:CellularComponent | statistical_association | data_analysis_pipeline | Source CTD data are based on an enrichment analysis of Gene Ontology (GO) terms annotated to all genes that interact with a particular chemical - to derive Chemical-GO Term associations. This analysis is performed by a CTD automated analysis pipeline. The CTD record used to create this Translator edge reports an overrepresented GO term associated with a particular chemical - as indicated by a sufficiently low p-value from the analysis. This hints but does not directly demonstrate that the chemical may impact the process, activity, or cellular component the GO term represents. The indirect, statistical basis of this relationship is best represented using the 'associated_with' predicate in Biolink. | |
| biolink:ChemicalEntity, biolink:MolecularMixture, biolink:SmallMolecule | biolink:Pathway | statistical_association | data_analysis_pipeline | Source CTD data are based on an enrichment analysis of Pathways annotated to all genes that interact with a particular chemical - to derive Chemical-Pathway associations. This analysis is performed by a CTD automated analysis pipeline. The CTD record used to create this Translator edge reports an overrepresented Pathway associated with a particular chemical - as indicated by a sufficiently low p-value from the analysis. This hints but does not directly demonstrate that the chemical may impact the Pathway. The indirect, statistical basis of this relationship is best represented using the 'associated_with' predicate in Biolink. |
Node Types¶
| Node Category | Source Identifier Types | Additional Notes |
|---|---|---|
| biolink:ChemicalEntity | MeSH | CTD identifies every chemical with a MeSH identifier, and the ingest emits them all as biolink:ChemicalEntity. Most resolve to more specific Biolink categories during normalization - the majority to biolink:SmallMolecule, with mixtures (common among the environmental stressors in the exposure data) resolving to biolink:MolecularMixture or biolink:ComplexMolecularMixture. |
| biolink:SmallMolecule | MeSH | |
| biolink:MolecularMixture | MeSH | |
| biolink:ComplexMolecularMixture | MeSH | |
| biolink:Disease | MeSH, OMIM | |
| biolink:PhenotypicFeature | GO | CTD reports "phenotypes" as GO terms rather than as disease-like phenotypic features, so these nodes carry GO identifiers and normalize to process/activity categories. |
| biolink:Gene | NCBIGene | |
| biolink:BiologicalProcess | GO | |
| biolink:MolecularActivity | GO | |
| biolink:CellularComponent | GO | |
| biolink:Pathway | KEGG.PATHWAY, REACT |
Future Modeling Considerations¶
predicates: Revisit 'correlated_with' and 'treats_or_studied_or_applied_to_treat' predicates if/when we refactor modeling or conventions here.
other: May want to improve ui_explanations for the chemical-gene interactions edge types, once we understand how the UI will stitch together / display predicates and qualifier values for these edges.
spoq_pattern: CTD_pheno_term_ixns and CTD_chem_gene_ixns both have lots of interactions that involve three entities, like A causes B which results in B causing C. These are hard to model on a single edge, so we currently drop them, but we may be able to extract single edges from them.
qualifiers: (related to the consideration above) The CTD_pheno_term_ixns.tsv that links chemicals to phenotypes (post-composed from GO term + direction) includes a column indicating gene(s) mediating the phenotype. Consider adding this info (perhaps using a new statement-level qualifier such as 'gene context qualifier', or 'mediating gene qualifier'.)
qualifiers: The chemical-gene sensitivity edges (from CTD's "response to substance" interactions) are emitted as a generic biolink:Association, because no Biolink chemical-gene association class accepts the (affects|increases|decreases)_sensitivity_to predicates. As a result they cannot carry the species context qualifier that the other chemical-gene edges carry, even though the source record provides it. Revisit if the Biolink association hierarchy gains a class that accepts these predicates.
edge_properties: If the inferred Chemical-Disease associations are ingested in a future iteration (see the ingest content considerations above), they would need an edge property to carry CTD's inference score. Revisit the use of 'has_confidence_score' for this if/when we refactor that part of the Biolink Model.
Provenance Information¶
Contributors: - Kevin Schaper: code author - Evan Morris: code author - Sierra Moxon: code support - Vlado Dancik: code support, domain expertise - Matthew Brush: data modeling, domain expertise
Artifacts: - Ingest Survey: https://docs.google.com/spreadsheets/d/1R9z-vywupNrD_3ywuOt_sntcTrNlGmhiUWDXUdkPVpM/edit?gid=0#gid=0 - KGX Summary Report of phase 2 ingests: https://docs.google.com/spreadsheets/d/1IVpkL0tFyk6U7c3tKFKKOlOhtJdMyZlI-1jn69LT0Rs/edit?gid=663403777#gid=663403777