Information content differentiates enhancers from silencers in mouse photoreceptors

RyanZFriedman1 2
DavidMGranas1 2
ConnieAMyers3
JosephCCorbo3
BarakACohen1 2
MichaelAWhite[email protected]1 2

Edison Family Center for Genome Sciences and Systems Biology, Washington University School of Medicine
St. LouisUnited States
Department of Genetics, Washington University School of Medicine
St. LouisUnited States
Department of Pathology and Immunology, Washington University School of Medicine
St LouisUnited States

2021-09-06

Research Article

Computational and Systems Biology
Genetics and Genomics

enhancers
silencers
information theory
massively parallel reporter assays
Mouse

publisher-id67403
doi10.7554/eLife.67403
elocation-ide67403

Abstract

Enhancers and silencers often depend on the same transcription factors (TFs) and are conflated in genomic assays of TF binding or chromatin state. To identify sequence features that distinguish enhancers and silencers, we assayed massively parallel reporter libraries of genomic sequences targeted by the photoreceptor TF cone-rod homeobox (CRX) in mouse retinas. Both enhancers and silencers contain more TF motifs than inactive sequences, but relative to silencers, enhancers contain motifs from a more diverse collection of TFs. We developed a measure of information content that describes the number and diversity of motifs in a sequence and found that, while both enhancers and silencers depend on CRX motifs, enhancers have higher information content. The ability of information content to distinguish enhancers and silencers targeted by the same TF illustrates how motif context determines the activity of cis-regulatory sequences.

Introduction

Active cis-regulatory sequences in the genome are characterized by accessible chromatin and specific histone modifications, which reflect the action of DNA-binding transcription factors (TFs) that recognize specific sequence motifs and recruit chromatin-modifying enzymes 44Klemm et al.2019. These epigenetic hallmarks of active chromatin are routinely used to train machine learning models that predict cis-regulatory sequences, based on the assumption that such epigenetic marks are reliable predictors of genuine cis-regulatory sequences 13Ernst and Kellis201219Ghandi et al.201427Hoffman et al.201241Kelley et al.201650Lee et al.201177Sethi et al.202090Zhou and Troyanskaya2015. However, results from functional assays show that many predicted cis-regulatory sequences exhibit little or no cis-regulatory activity. Typically, 50% or more of predicted cis-regulatory sequences fail to drive expression in massively parallel reporter assays (MPRAs) 58Moore et al.202048Kwasnieski et al.2014, indicating that an active chromatin state is not sufficient to reliably identify cis-regulatory sequences.

Another challenge is that enhancers and silencers are difficult to distinguish by chromatin accessibility or epigenetic state 11Doni Jayavelu et al.202020Gisselbrecht et al.202062Pang and Snyder202066Petrykowska et al.200876Segert et al.2021, and thus computational predictions of _cis-_regulatory sequences often do not differentiate between enhancers and silencers. Silencers are often enhancers in other cell types 5Brand et al.198711Doni Jayavelu et al.202020Gisselbrecht et al.202030Huang et al.202137Jiang et al.199361Ngan et al.202062Pang and Snyder2020, reside in open chromatin 11Doni Jayavelu et al.202029Huang et al.201930Huang et al.202162Pang and Snyder2020, sometimes bear epigenetic marks of active enhancers 14Fan et al.201630Huang et al.2021, and can be bound by TFs that also act on enhancers in the same cell type 1Alexandre and Vincent200321Grass et al.200330Huang et al.202135Iype et al.200437Jiang et al.199352Liu et al.201453Martínez-Montañés et al.201365Peng et al.200569Rachmin et al.201570Rister et al.201580Stampfel et al.201585White et al.2013. As a result, enhancers and silencers share similar sequence features, and understanding how they are distinguished in a particular cell type remains an important challenge 76Segert et al.2021.

The TF cone-rod homeobox (CRX) controls selective gene expression in a number of different photoreceptor and bipolar cell types in the retina 6Chen et al.199717Freund et al.199718Furukawa et al.199760Murphy et al.2019. These cell types derive from the same progenitor cell population 45Koike et al.200783Wang et al.2014, but they exhibit divergent, CRX-directed transcriptional programs 9Corbo et al.201025Hennig et al.200831Hughes et al.201760Murphy et al.2019. CRX cooperates with cell type-specific co-factors to selectively activate and repress different genes in different cell types and is required for differentiation of rod and cone photoreceptors 7Chen et al.200523Hao et al.201225Hennig et al.200828Hsiau et al.200734Irie et al.201543Kimura et al.200051Lerner et al.200555Mears et al.200156Mitton et al.200060Murphy et al.201965Peng et al.200575Sanuki et al.201079Srinivas et al.2006. However, the sequence features that define CRX-targeted enhancers vs. silencers in the retina are largely unknown.

We previously found that a significant minority of CRX-bound sequences act as silencers in an MPRA conducted in live mouse retinas 85White et al.2013, and that silencer activity requires CRX 86White et al.2016. Here, we extend our analysis by testing thousands of additional candidate cis-regulatory sequences. We show that while regions of accessible chromatin and CRX binding exhibit a range of cis-regulatory activity, enhancers and silencers contain more TF motifs than inactive sequences, and that enhancers are distinguished from silencers by a higher diversity of TF motifs. We capture the differences between these sequence classes with a new metric, motif information content (Boltzmann entropy), that considers only the number and diversity of TF motifs in a candidate cis-regulatory sequence. Our results suggest that CRX-targeted enhancers are defined by a flexible regulatory grammar and demonstrate how differences in motif information content encode functional differences between genomic loci with similar chromatin states.

# Setup imports for analysis
import os
import sys
import itertools

import numpy as np
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
from mpl_toolkits.axes_grid1 import make_axes_locatable
from scipy import stats
from sklearn.feature_selection import RFE, RFECV
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold
from pybedtools import BedTool
from IPython.display import display
import logomaker

sys.path.insert(0, "utils")
from utils import fasta_seq_parse_manip, gkmsvm, modeling, plot_utils, predicted_occupancy, quality_control, sequence_annotation_processing

data_dir = os.path.join("Data")
figures_dir = os.path.join("Figures")

# Load in all sequences
all_seqs = fasta_seq_parse_manip.read_fasta(os.path.join(data_dir, "library1And2.fasta"))
# Drop scrambled sequences -- we don't need them for any analysis
all_seqs = all_seqs[~(all_seqs.index.str.contains("scr"))]

plot_utils.set_manuscript_params()

Results

We tested the activities of 4844 putative CRX-targeted cis-regulatory sequences (CRX-targeted sequences) by MPRA in live retinas. The MPRA libraries consist of 164 bp genomic sequences centered on the best match to the CRX position weight matrix (PWM) 49Lee et al.2010 whenever a CRX motif is present, and matched sequences in which all CRX motifs were abolished by point mutation (Materials and methods). The MPRA libraries include 3299 CRX-bound sequences identified by ChIP-seq in the adult retina 9Corbo et al.2010 and 1545 sequences that do not have measurable CRX binding in the adult retina but reside in accessible chromatin in adult photoreceptors 31Hughes et al.2017 and have the H3K27ac enhancer mark in postnatal day 14 (P14) retina 72Ruzycki et al.2018 (‘ATAC-seq peaks’). We split the sequences across two plasmid libraries, each of which contained the same 150 scrambled sequences as internal controls (Supplementary files 1 and 2). We cloned sequences upstream of the rod photoreceptor-specific Rhodopsin (Rho) promoter and a DsRed reporter gene, electroporated libraries into explanted mouse retinas at P0 in triplicate, harvested the retinas at P8, and then sequenced the RNA and input DNA plasmid pool. The data is highly reproducible across replicates (R² > 0.96, Figure 1—figure supplement 1). After activity scores were calculated and normalized to the basal Rho promoter, the two libraries were well calibrated and merged together (two-sample Kolmogorov-Smirnov test p = 0.09, Figure 1—figure supplement 2, Supplementary file 3, and Materials and methods).

# Process data for the Rho promoter: convert counts into activity scores for each sequence
library_names = ["library1", "library2"]
rho_activity_data = {} # {library name: pd.DataFrame}
barcode_count_dir = os.path.join(data_dir, "Rhodopsin")

for library in library_names:
    print(f"Processing data for {library} with the Rho promoter...")
    # File names
    barcode_count_files = [
        os.path.join(barcode_count_dir, f"{library}{sample}.counts")
        for sample in ["Plasmid", "Rna1", "Rna2", "Rna3"]
    ]
    
    # Masks and metadata for downstream functions
    sample_labels = np.array(["DNA", "RNA1", "RNA2", "RNA3"])
    sample_rna_mask = np.array([False, True, True, True])
    rna_labels = sample_labels[sample_rna_mask]
    dna_labels = sample_labels[np.logical_not(sample_rna_mask)]
    n_samples = len(sample_labels)
    n_rna_samples = len(rna_labels)
    n_dna_samples = len(dna_labels)
    n_barcodes_per_sequence = 3
    
    # Read in the barcode counts
    print("Reading in barcode counts.")
    all_sample_counts_df = quality_control.read_bc_count_files(barcode_count_files, sample_labels)
    display(all_sample_counts_df.head())
    
    # Remove barcodes that are detection-limited.
    # Barcodes below the DNA cutoff are NaN (because they are missing from the input plasmid pool)
    # Barcodes below any of the RNA cutoffs are zero in all replicates
    print("Removing detection-limited barcodes and normalizing to counts per million.")
    cutoffs = [10, 5, 5, 5]
    threshold_sample_counts_df = quality_control.filter_low_counts(all_sample_counts_df, sample_labels, cutoffs,
                                                                   dna_labels=dna_labels, bc_per_seq=n_barcodes_per_sequence)
    display(threshold_sample_counts_df.head())

    # Normalize RNA barcode counts by plasmid barcode counts
    print("Normalizing RNA to DNA.")
    normalized_sample_counts_df = quality_control.normalize_rna_by_dna(threshold_sample_counts_df, rna_labels, dna_labels)
    # Drop DNA
    barcode_sample_counts_df = normalized_sample_counts_df.drop(columns=dna_labels)
    
    # Average across barcodes
    print("Averaging across barcodes within a replicate.")
    activity_replicate_df = quality_control.average_barcodes(barcode_sample_counts_df)
    display(activity_replicate_df.head())
    
    # Basal-normalize, average across replicates, do statistics
    print("Normalizing to the basal Rho promoter.")
    sequence_expression_df = quality_control.basal_normalize(activity_replicate_df, "BASAL")
    print("Computing p-values for the null hypothesis that a sequence is no different than the basal promoter alone.")
    sequence_expression_df["expression_pvalue"] = quality_control.log_ttest_vs_basal(activity_replicate_df, "BASAL")
    sequence_expression_df["expression_qvalue"] = modeling.fdr(sequence_expression_df["expression_pvalue"])
    print(f"Done processing data!")
    display(sequence_expression_df.head())
    
    rho_activity_data[library] = sequence_expression_df

	label	DNA	RNA1	RNA2	RNA3
AACAACAAG	chr16-87432635-87432799_CPPQ_scrambled	3019	148	325	97
AACAACCGC	chr4-119112319-119112483_CPPE_WT	4117	24493	25950	23406
AACAACGGG	chr7-128854234-128854398_UPCE_WT	86	76	39	233
AACAACTAC	chr4-138107597-138107761_UPPE_WT	827	926	857	659
AACAACTGT	chr5-31298508-31298672_CPPE_WT	7170	492	392	149

	label	DNA	RNA1	RNA2	RNA3
AACAACAAG	chr16-87432635-87432799_CPPQ_scrambled	73.436588	4.307406	7.418047	2.561422
AACAACCGC	chr4-119112319-119112483_CPPE_WT	100.145224	712.846538	592.302519	618.068596
AACAACGGG	chr7-128854234-128854398_UPCE_WT	2.091933	2.211911	0.890166	6.152695
AACAACTAC	chr4-138107597-138107761_UPPE_WT	20.116614	26.95039	19.560819	17.401829
AACAACTGT	chr5-31298508-31298672_CPPE_WT	174.408855	14.319214	8.947306	3.934556

	RNA1	RNA2	RNA3
BASAL	0.331679	0.306512	0.277308
chr1-104768570-104768734_UPCQ_MUT-allCrxSites	1.005172	0.826315	0.930872
chr1-104768570-104768734_UPCQ_WT	1.114088	1.080287	1.091619
chr1-106008207-106008371_CPPE_MUT-allCrxSites	1.180305	1.094909	0.798394
chr1-106008207-106008371_CPPE_WT	0.441799	0.533383	0.86899

	expression	expression_std	expression_reps	expression_pvalue	expression_qvalue
chr1-104768570-104768734_UPCQ_MUT-allCrxSites	3.027744	0.330482	3	0.000139	0.000749
chr1-104768570-104768734_UPCQ_WT	3.606621	0.297412	3	0.001206	0.003548
chr1-106008207-106008371_CPPE_MUT-allCrxSites	3.336604	0.396284	3	0.003039	0.007388
chr1-106008207-106008371_CPPE_WT	2.068611	0.944664	3	0.080583	0.103242
chr1-106171416-106171580_CSPE_scrambled	1.439587	0.579277	3	0.27973	0.312931

	label	DNA	RNA1	RNA2	RNA3
AACAACAAG	chr7-141291911-141292075_UPPP_MUT-allCrxSites	132	0	1	1
AACAACGTT	chr19-16380352-16380516_CPPN_MUT-allCrxSites	1779	36	17	46
AACAACTAC	chr1-44147572-44147736_UPPP_MUT-allCrxSites	2928	433	802	510
AACAACTCG	chr12-116230818-116230982_CPPE_WT	2822	3043	2967	3013
AACAACTGT	chr5-65391346-65391510_CPPP_MUT-allCrxSites	1810	1572	2281	1559

	label	DNA	RNA1	RNA2	RNA3
AACAACAAG	chr7-141291911-141292075_UPPP_MUT-allCrxSites	3.144868	0	0	0
AACAACGTT	chr19-16380352-16380516_CPPN_MUT-allCrxSites	42.384243	0.933407	0.406204	1.301935
AACAACTAC	chr1-44147572-44147736_UPPP_MUT-allCrxSites	69.758888	11.226812	19.16328	14.434499
AACAACTCG	chr12-116230818-116230982_CPPE_WT	67.233464	78.898818	70.894577	85.276757
AACAACTGT	chr5-65391346-65391510_CPPP_MUT-allCrxSites	43.12281	40.758772	54.503043	44.124283

	RNA1	RNA2	RNA3
BASAL	0.196778	0.218638	0.236666
chr1-10229074-10229238_CPPE_MUT-allCrxSites	7.325586	5.922791	6.286389
chr1-10229074-10229238_CPPE_WT	6.418129	5.188716	4.97623
chr1-106171416-106171580_CSPE_MUT-shape	0.282047	0.264416	0.290612
chr1-106171416-106171580_CSPE_WT	0.260469	0.27625	0.212923

	expression	expression_std	expression_reps	expression_pvalue	expression_qvalue
chr1-10229074-10229238_CPPE_MUT-allCrxSites	30.293101	6.01123	3	0.000003	0.000128
chr1-10229074-10229238_CPPE_WT	25.791454	6.063103	3	0.000019	0.000167
chr1-106171416-106171580_CSPE_MUT-shape	1.290214	0.124284	3	0.023905	0.031469
chr1-106171416-106171580_CSPE_WT	1.162281	0.229405	3	0.226254	0.246199
chr1-106171416-106171580_CSPE_scrambled	1.995027	0.380942	3	0.012703	0.018175

# Now process data for the Polylinker (experiment is in Fig 4, but it is easier to process the data here)
# Process data for the Rho promoter: convert counts into activity scores for each sequence
library_names = ["library1", "library2"]
polylinker_activity_data = {} # {library name: pd.DataFrame}
barcode_count_dir = os.path.join(data_dir, "Polylinker")

for library in library_names:
    print(f"Processing data for {library} with the Polylinker...")
    # File names
    barcode_count_files = [
        os.path.join(barcode_count_dir, f"{library}{sample}.counts")
        for sample in ["Plasmid", "Rna1", "Rna2", "Rna3"]
    ]
    
    # Masks and metadata for downstream functions
    sample_labels = np.array(["DNA", "RNA1", "RNA2", "RNA3"])
    sample_rna_mask = np.array([False, True, True, True])
    rna_labels = sample_labels[sample_rna_mask]
    dna_labels = sample_labels[np.logical_not(sample_rna_mask)]
    n_samples = len(sample_labels)
    n_rna_samples = len(rna_labels)
    n_dna_samples = len(dna_labels)
    n_barcodes_per_sequence = 3
    
    # Read in the barcode counts
    print("Reading in barcode counts.")
    all_sample_counts_df = quality_control.read_bc_count_files(barcode_count_files, sample_labels)
    display(all_sample_counts_df.head())
    
    # Remove barcodes that are detection-limited.
    print("Removing barcodes missing from the DNA pool and normalizing to counts per million.")
    cutoffs_dna_only = [50, 0, 0, 0]
    # Barcodes below the DNA cutoff are NaN (because they are missing from the input plasmid pool)
    # Barcodes below any of the RNA cutoffs are zero in all replicates
    print("Removing detection-limited barcodes and normalizing to counts per million.")
    threshold_sample_counts_df = quality_control.filter_low_counts(all_sample_counts_df, sample_labels, cutoffs_dna_only,
                                                                   dna_labels=dna_labels, bc_per_seq=n_barcodes_per_sequence)
    print("Now removing RNA barcodes missing from any replicate.")
    cutoffs_rna_cpm = [0, 8, 8, 8]
    threshold_sample_counts_df = quality_control.filter_low_counts(threshold_sample_counts_df, sample_labels, cutoffs_rna_cpm,
                                                                  dna_labels=dna_labels, bc_per_seq=n_barcodes_per_sequence, cpm_normalize=False)
    display(threshold_sample_counts_df.head())

    # Normalize RNA barcode counts by plasmid barcode counts
    print("Normalizing RNA to DNA.")
    normalized_sample_counts_df = quality_control.normalize_rna_by_dna(threshold_sample_counts_df, rna_labels, dna_labels)
    # Drop DNA
    barcode_sample_counts_df = normalized_sample_counts_df.drop(columns=dna_labels)
    
    # Average across barcodes
    print("Averaging across barcodes within a replicate.")
    activity_replicate_df = quality_control.average_barcodes(barcode_sample_counts_df)
    display(activity_replicate_df.head())
    
    # Drop "basal" and average across replicates
    print("Removing the 'basal' promoter (Polylinker) and averaging across replicates. No statistical analysis is performed here.")
    activity_replicate_df = activity_replicate_df.drop(index="BASAL")
    sequence_expression_df = activity_replicate_df.apply(lambda x: pd.Series({"expression": x.mean(), "expression_SEM": x.sem()}), axis=1)
    print(f"Done processing data!")
    display(sequence_expression_df.head())
    
    polylinker_activity_data[library] = sequence_expression_df

	label	DNA	RNA1	RNA2	RNA3
AACAACAAG	chr16-87432635-87432799_CPPQ_scrambled	987	2	3	10
AACAACCGC	chr4-119112319-119112483_CPPE_WT	1326	4963	4554	17827
AACAACGGG	chr7-128854234-128854398_UPCE_WT	35	0	0	2
AACAACTAC	chr4-138107597-138107761_UPPE_WT	5	8	6	4
AACAACTGT	chr5-31298508-31298672_CPPE_WT	5007	934	993	575

	label	DNA	RNA1	RNA2	RNA3
AACAACAAG	chr16-87432635-87432799_CPPQ_scrambled	48.214705	0	0	0
AACAACCGC	chr4-119112319-119112483_CPPE_WT	64.774771	238.306557	198.604223	639.087016
AACAACGGG	chr7-128854234-128854398_UPCE_WT	NaN	0	0	0
AACAACTAC	chr4-138107597-138107761_UPPE_WT	NaN	0	0	0
AACAACTGT	chr5-31298508-31298672_CPPE_WT	244.590708	44.847537	43.305664	20.613397

	RNA1	RNA2	RNA3
BASAL	0.742818	0.983263	1.267636
chr1-104768570-104768734_UPCQ_MUT-allCrxSites	0	0	0
chr1-104768570-104768734_UPCQ_WT	0	0	0
chr1-106008207-106008371_CPPE_MUT-allCrxSites	0	0	0
chr1-106008207-106008371_CPPE_WT	0	0	0

	expression	expression_SEM
chr1-104768570-104768734_UPCQ_MUT-allCrxSites	0	0
chr1-104768570-104768734_UPCQ_WT	0	0
chr1-106008207-106008371_CPPE_MUT-allCrxSites	0	0
chr1-106008207-106008371_CPPE_WT	0	0
chr1-106171416-106171580_CSPE_scrambled	0	0

	label	DNA	RNA1	RNA2	RNA3
AACAACAAG	chr7-141291911-141292075_UPPP_MUT-allCrxSites	3	20	15	21
AACAACGTT	chr19-16380352-16380516_CPPN_MUT-allCrxSites	990	10	9	10
AACAACTAC	chr1-44147572-44147736_UPPP_MUT-allCrxSites	1056	2	4	3
AACAACTCG	chr12-116230818-116230982_CPPE_WT	7	4	6	0
AACAACTGT	chr5-65391346-65391510_CPPP_MUT-allCrxSites	1653	1441	9	4695

	label	DNA
AACAACAAG	chr7-141291911-141292075_UPPP_MUT-allCrxSites	NaN
AACAACGTT	chr19-16380352-16380516_CPPN_MUT-allCrxSites	38.377926
AACAACTAC	chr1-44147572-44147736_UPPP_MUT-allCrxSites	40.936454
AACAACTCG	chr12-116230818-116230982_CPPE_WT	NaN
AACAACTGT	chr5-65391346-65391510_CPPP_MUT-allCrxSites	64.079506

	RNA1	RNA2	RNA3
BASAL	0	0	0
chr1-10229074-10229238_CPPE_MUT-allCrxSites	1.486824	0.405204	1.305344
chr1-10229074-10229238_CPPE_WT	0	0	0
chr1-106171416-106171580_CSPE_MUT-shape	0	0	0
chr1-106171416-106171580_CSPE_WT	0	0	0

	expression	expression_SEM
chr1-10229074-10229238_CPPE_MUT-allCrxSites	1.06579	0.334422
chr1-10229074-10229238_CPPE_WT	0	0
chr1-106171416-106171580_CSPE_MUT-shape	0	0
chr1-106171416-106171580_CSPE_WT	0	0
chr1-106171416-106171580_CSPE_scrambled	0	0

Reproducibility of massively parallel reporter assay (MPRA) measurements.

Each row represents a different library and experiment. For each column, the first replicate in the title is the x-axis and the second replicate is the y-axis.

Calibration of massively parallel reporter assay (MPRA) libraries with the *Rho* promoter.

Probability density histogram of the same 150 scrambled sequences in two libraries after normalizing to the basal *Rho* promoter.

# Join and annotate all data
print("Joining together data from the two libraries with the Rho promoter.")
color_mapping = {
    "Strong enhancer": "#1f78b4",
    "Weak enhancer": "#a6cee3",
    "Inactive": "#33a02c",
    "Silencer": "#e31a1c",
    np.nan: "grey"
}

# Join the libraries and add a pseudocount to take log2
rho_df = library1_rho_df.append(library2_rho_df)
rho_pseudocount = 1e-3
rho_df["expression_log2"] = np.log2(rho_df["expression"] + rho_pseudocount)

# Define cutoff for a strong enhancer based on scrambled sequences
print("Annotating sequences as strong enhancer, weak enhancer, inactive, silencer, or ambiguous.")
scrambled_mask = rho_df.index.str.contains("scrambled")
scrambled_df = rho_df[scrambled_mask]
scrambled_df = scrambled_df[scrambled_df["expression"].notna()]
strong_cutoff = scrambled_df["expression_log2"].quantile(0.95)
print(f"Cutoff to call something a strong enhancer: activity is above {strong_cutoff:.2f}")

# Drop scrambled sequences
rho_df = rho_df[~scrambled_mask]

# Helper function to label and color a sequence
def label_color_sequence(row, alpha=0.05, strong_cutoff=strong_cutoff, inactive_cutoff=1, color_mapping=color_mapping):
    expr_log2 = row["expression_log2"]
    qval = row["expression_qvalue"]
    # Inactive
    if (np.abs(expr_log2) <= inactive_cutoff) & (qval >= alpha):
        group = "Inactive"
    # Silencer
    elif (expr_log2 < -inactive_cutoff) & ((qval < alpha) | (row["expression"] == 0)):
        group = "Silencer"
    # Enhancer
    elif (expr_log2 > inactive_cutoff) & (qval < alpha):
        # Strong
        if expr_log2 > strong_cutoff:
            group = "Strong enhancer"
        # Weak
        else:
            group = "Weak enhancer"
    # Ambiguous
    else:
        group = np.nan
    
    color = color_mapping[group]
    return pd.Series({"group_name": group, "plot_color": color})

# Annotate both WT and MUT sequences
rho_df = rho_df.join(rho_df.apply(label_color_sequence, axis=1))
rho_df["group_name"] = sequence_annotation_processing.to_categorical(rho_df["group_name"])

# Now do Polylinker data
library1_poly_df = polylinker_activity_data["library1"]
library2_poly_df = polylinker_activity_data["library2"]
print("Joining together data from the two libraries with the Polylinker promoter and annotate for autonomous activity.")
poly_df = library1_poly_df.append(library2_poly_df)
poly_pseudocount = 1e-2
poly_df["expression_log2"] = np.log2(poly_df["expression"] + poly_pseudocount)
poly_df["autonomous_activity"] = (poly_df["expression_log2"] > 0)

# Compute effect of mutating CRX motifs in the presence of the Rho promoter.
print("Computing the effect size upon mutating CRX motifs in the presence of the Rho promoter.")
print("This is for Figure 5, but it is easier to do it here.")
wt_mask = rho_df.index.str.contains("_WT$")
mut_mask = rho_df.index.str.contains("_MUT-allCrxSites$")

# Add variant info as a column, then trim it off the index
rho_df_no_variant_df = rho_df.copy()
rho_df_no_variant_df["variant"] = rho_df_no_variant_df.index.str.split("_").str[2:].str.join("_")
rho_df_no_variant_df = sequence_annotation_processing.remove_mutations_from_seq_id(rho_df_no_variant_df)

# Separate out WT and MUT, then join them together on the same row
wt_df = rho_df_no_variant_df[wt_mask]
mut_df = rho_df_no_variant_df[mut_mask]
wt_vs_mut_rho_df = wt_df.join(mut_df, lsuffix="_WT", rsuffix="_MUT")
wt_vs_mut_rho_df["wt_vs_mut_log2"] = wt_vs_mut_rho_df["expression_log2_WT"] - wt_vs_mut_rho_df["expression_log2_MUT"]

# Compute parameters for lognormal distribution to do stats
wt_cov = wt_vs_mut_rho_df["expression_std_WT"] / wt_vs_mut_rho_df["expression_WT"]
wt_log_mean = np.log(wt_vs_mut_rho_df["expression_WT"] / np.sqrt(wt_cov**2 + 1))
wt_log_std = np.sqrt(np.log(wt_cov**2 + 1))
mut_cov = wt_vs_mut_rho_df["expression_std_MUT"] / wt_vs_mut_rho_df["expression_MUT"]
mut_log_mean = np.log(wt_vs_mut_rho_df["expression_MUT"] / np.sqrt(mut_cov**2 + 1))
mut_log_std = np.sqrt(np.log(mut_cov**2 + 1))

# Do t-tests and FDR
wt_vs_mut_rho_df["wt_vs_mut_pvalue"] = stats.ttest_ind_from_stats(wt_log_mean, wt_log_std, wt_vs_mut_rho_df["expression_reps_WT"], mut_log_mean, mut_log_std, wt_vs_mut_rho_df["expression_reps_MUT"], equal_var=False)[1]
wt_vs_mut_rho_df["wt_vs_mut_qvalue"] = modeling.fdr(wt_vs_mut_rho_df["wt_vs_mut_pvalue"])

# Pull out WT polylinker measurements
print("Joining Rho and Polylinker data together.")
poly_wt_df = poly_df.copy()
poly_wt_df = poly_wt_df[poly_wt_df.index.str.contains("WT")]

# Drop the variant ID
poly_wt_df = poly_wt_df.rename(index=lambda x: x[:-3], columns={"expression": "expression_POLY", "expression_SEM": "expression_SEM_POLY", "expression_log2": "expression_log2_POLY"})

# Join with Rho
activity_df = wt_vs_mut_rho_df.join(poly_wt_df)

print("Annotating sequences for binding patterns.")
# Get info on CRX binding from the seq ID strings
activity_df["crx_bound"] = activity_df.index.str.contains("_C...$")

# Read in BED files
library_bed = BedTool(os.path.join(data_dir, "library1And2.bed"))
nrl_chip_bed = BedTool(os.path.join("Data", "Downloaded", "ChIP", "nrlPeaksMm10.bed"))
mef2d_chip_bed = BedTool(os.path.join("Data", "Downloaded", "ChIP", "mef2dPeaksMm10.bed"))

# Get binding patterns for NRL and MEF2D
library_nrl_bound_df = library_bed.intersect(nrl_chip_bed, wa=True).to_dataframe()
activity_df["nrl_bound"] = activity_df.index.isin(library_nrl_bound_df["name"])

library_mef2d_bound_df = library_bed.intersect(mef2d_chip_bed, wa=True).to_dataframe()
activity_df["mef2d_bound"] = activity_df.index.isin(library_mef2d_bound_df["name"])

# Helper function to "reverse one hot encode" binding patterns
def annotate_binding(row):
    crx, nrl, mef2d = row[["crx_bound", "nrl_bound", "mef2d_bound"]]
    if crx:
        if nrl:
            if mef2d:
                return "All three"
            else:
                return "CRX+NRL"
        elif mef2d:
            return "CRX+MEF2D"
        else:
            return "CRX only"
    elif nrl:
        if mef2d:
            return "NRL+MEF2D"
        else:
            return "NRL only"
    elif mef2d:
        return "MEF2D only"
    else:
        return "No binding"

activity_df["binding_group"] = activity_df.apply(annotate_binding, axis=1)
print("Done processing and annotating data. This table corresponds to Supplementary file 3.")
display(activity_df.head())

Joining together data from the two libraries with the Rho promoter.
Annotating sequences as strong enhancer, weak enhancer, inactive, silencer, or ambiguous.
Cutoff to call something a strong enhancer: activity is above 2.84
Joining together data from the two libraries with the Polylinker promoter and annotate for autonomous activity.
Computing the effect size upon mutating CRX motifs in the presence of the Rho promoter.
This is for Figure 5, but it is easier to do it here.
Joining Rho and Polylinker data together.
Annotating sequences for binding patterns.

/home/ryan/miniconda/envs/bclab/lib/python3.6/site-packages/scipy/stats/_distn_infrastructure.py:879: RuntimeWarning: invalid value encountered in greater
  return (self.a < x) & (x < self.b)
/home/ryan/miniconda/envs/bclab/lib/python3.6/site-packages/scipy/stats/_distn_infrastructure.py:879: RuntimeWarning: invalid value encountered in less
  return (self.a < x) & (x < self.b)
/home/ryan/miniconda/envs/bclab/lib/python3.6/site-packages/scipy/stats/_distn_infrastructure.py:1821: RuntimeWarning: invalid value encountered in less_equal
  cond2 = cond0 & (x <= self.a)

Done processing and annotating data. This table corresponds to Supplementary file 3.

Strong enhancers and silencers have high CRX motif content

The cis-regulatory activities of CRX-targeted sequences vary widely (Figure 1a). We defined enhancers and silencers as those sequences that have statistically significant activity that is at least twofold above or below the activity of the basal Rho promoter (Welch’s t-test, Benjamini-Hochberg false discovery rate (FDR) q < 0.05, Supplementary file 3). We defined inactive sequences as those whose activity is both within a twofold change of basal activity and not significantly different from the basal Rho promoter. We further stratified enhancers into strong and weak enhancers based on whether or not they fell above the 95th percentile of scrambled sequences. Using these criteria, 22% of CRX-targeted sequences are strong enhancers, 28% are weak enhancers, 19% are inactive, and 17% are silencers; the remaining 13% were considered ambiguous and removed from further analysis. To test whether these sequences function as CRX-dependent enhancers and silencers in the genome, we examined genes differentially expressed in Crx^-/- retina 71Roger et al.2014. Genes that are de-repressed are more likely to be near silencers (Fisher’s exact test p = 0.001, odds ratio = 2.1, n = 206) and genes that are down-regulated are more likely to be near enhancers (Fisher’s exact test p = 0.02, odds ratio = 1.5, n = 344, Materials and methods), suggesting that our reporter assay identified sequences that act as enhancers and silencers in the genome. We sought to identify features that would accurately classify these different classes of sequences.

# Calculate predicted occupancy of all TFs
print("Computing predicted occupancy of 8 TFs on every WT and mutant sequence. This might take 2-3 minutes.")

# Load in PWMs
pwms = predicted_occupancy.read_pwm_files(os.path.join("Data", "Downloaded", "Pwm", "photoreceptorAndEnrichedMotifs.meme"))
pwms = pwms.rename(lambda x: x.split("_")[0])
# Reverse compliment RAX for display purposes
rax = pwms["RAX"].copy()
rax = rax[::-1].reset_index(drop=True)
rax_rc = rax.copy()
rax_rc["A"] = rax["T"]
rax_rc["C"] = rax["G"]
rax_rc["G"] = rax["C"]
rax_rc["T"] = rax["A"]
pwms["RAX"] = rax_rc
motif_len = pwms.apply(len)
ewms = pwms.apply(predicted_occupancy.ewm_from_letter_prob).apply(predicted_occupancy.ewm_to_dict)
mu = 9

# Do predicted occupancy scans
occupancy_df = predicted_occupancy.all_seq_total_occupancy(all_seqs, ewms, mu, convert_ewm=False)
print("Done computing predicted occupancies. This corresponds to Supplementary table 4.")
display(occupancy_df.head())

# Separate out the WT sequences
wt_occupancy_df = occupancy_df[occupancy_df.index.str.contains("WT$")].copy()
wt_occupancy_df = sequence_annotation_processing.remove_mutations_from_seq_id(wt_occupancy_df)
wt_occupancy_df = wt_occupancy_df.loc[activity_df.index]
n_tfs = len(wt_occupancy_df.columns)

Computing predicted occupancy of 8 TFs on every WT and mutant sequence. This might take 2-3 minutes.
Done computing predicted occupancies. This corresponds to Supplementary table 4.

	CRX	GFI1	MAZ	MEF2D	NDF1	NRL	RORB	RAX
label
chr1-4357766-4357930_CPPP_WT	2.297972	0.187172	2.204502e-8	0.000001421229	3.064604e-7	1.001505	0.02370847	0.005755
chr1-4357766-4357930_CPPP_MUT-allCrxSites	0.239708	3.783122e-11	2.204502e-8	0.000001421229	3.064606e-7	1.411916	0.02340304	0.004416
chr1-73826292-73826456_CPPE_WT	2.290427	0.00639738	0.005577725	1.815852e-9	6.713635e-7	0.993418	0.0002922269	0.000004
chr1-73826292-73826456_CPPE_MUT-allCrxSites	0.29341	1.20373e-8	0.005577725	6.339047e-11	6.713632e-7	0.993414	1.23963e-7	0.000002
chr11-87108697-87108861_CPPP_WT	2.71847	0.6025624	2.74423e-12	0.000002986062	6.477337e-7	0.040965	0.00004672926	0.190641

print("Computing information content of sequences.")
entropy_df = occupancy_df.apply(predicted_occupancy.boltzmann_entropy, axis=1)
print("Done computing information content and related metrics. This corresponds to Supplementary table 5.")
display(entropy_df.head())

wt_entropy_df = entropy_df[entropy_df.index.str.contains("WT$")].copy()
wt_entropy_df = sequence_annotation_processing.remove_mutations_from_seq_id(wt_entropy_df)
wt_entropy_df = wt_entropy_df.loc[activity_df.index]

mut_entropy_df = entropy_df[entropy_df.index.str.contains("MUT")].copy()
mut_entropy_df = sequence_annotation_processing.remove_mutations_from_seq_id(mut_entropy_df)
mut_entropy_df = mut_entropy_df.loc[activity_df.index]

Computing information content of sequences.
Done computing information content and related metrics. This corresponds to Supplementary table 5.

	total_occupancy	diversity	entropy
label
chr1-4357766-4357930_CPPP_WT	3.516114	2	2.291861
chr1-4357766-4357930_CPPP_MUT-allCrxSites	1.679445	1	0.440493
chr1-73826292-73826456_CPPE_WT	3.296117	2	1.74337
chr1-73826292-73826456_CPPE_MUT-allCrxSites	1.292404	1	0.378922
chr11-87108697-87108861_CPPP_WT	3.552689	2	1.867968

Activity of putative *cis*-regulatory sequences with cone-rod homeobox (CRX) motifs.

(a) Volcano plot of activity scores relative to the *Rho* promoter alone. Sequences are grouped as strong enhancers (dark blue), weak enhancers (light blue), inactive (green), silencers (red), or ambiguous (gray). Horizontal line, false discovery rate (FDR) q = 0.05. Vertical lines, twofold above and below *Rho*. (b) Fraction of ChIP-seq and ATAC-seq peaks that belong to each activity group. (c) Predicted CRX occupancy of each activity group. Horizontal lines, medians; enh., enhancer. Numbers at top of (**b and c**) indicate n for groups.

group_name_WT	Silencer	Inactive	Weak enhancer	Strong enhancer
ATAC-seq	281	363	430	211
ChIP-seq	556	565	930	840

group_name_WT	Silencer	Inactive	Weak enhancer	Strong enhancer
ATAC-seq	0.218677	0.28249	0.33463	0.164202
ChIP-seq	0.192321	0.195434	0.321688	0.290557

	count	mean	std	min	25%	50%	75%	max
Silencer	837	2.822068	1.474613	0.013521	1.59851	2.724195	3.916786	8.028408
Inactive	928	2.232489	1.342345	0.001052	1.173444	2.048457	3.136282	6.759976
Weak enhancer	1360	2.216861	1.220496	0.000385	1.235126	2.11381	2.988673	7.801177
Strong enhancer	1051	2.53401	1.16946	0.003694	1.616414	2.490314	3.285321	7.3685

Neither CRX ChIP-seq-binding status nor DNA accessibility as measured by ATAC-seq strongly differentiates between these four classes (Figure 1b). Compared to CRX ChIP-seq peaks, ATAC-seq peaks that lack CRX binding in the adult retina are slightly enriched for inactive sequences (Fisher’s exact test p = 2 × 10^–7, odds ratio = 1.5) and slightly depleted for strong enhancers (Fisher’s exact test p = 1 × 10^–21, odds ratio = 2.2). However, sequences with ChIP-seq or ATAC-seq peaks span all four activity categories, consistent with prior reports that DNA accessibility and TF binding data are not sufficient to identify functional enhancers and silencers 11Doni Jayavelu et al.202029Huang et al.201930Huang et al.202162Pang and Snyder202085White et al.2013.

We examined whether the number and affinity of CRX motifs differentiate enhancers, silencers, and inactive sequences by computing the predicted CRX occupancy (i.e. expected number of bound molecules) for each sequence 85White et al.2013. Consistent with our previous work 86White et al.2016, both strong enhancers and silencers have higher predicted CRX occupancy than inactive sequences (Mann-Whitney U test, p = 6 × 10^–10 and 6 × 10^–17, respectively, Figure 1c), suggesting that total CRX motif content helps distinguish silencers and strong enhancers from inactive sequences. However, predicted CRX occupancy does not distinguish strong enhancers from silencers: a logistic regression classifier trained with fivefold cross-validation only achieves an area under the receiver operating characteristic (AUROC) curve of 0.548 ± 0.023 and an area under the precision recall (AUPR) curve of 0.571 ± 0.020 (Figure 2a and Figure 2—figure supplement 1). We thus sought to identify sequence features that distinguish strong enhancers from silencers.

# Prepare data for fitting models
# Mask to pull out the silencers and strong enhancers
silencer_modeling_mask = activity_df["group_name_WT"].str.contains("Strong|Silencer")
silencer_modeling_mask = silencer_modeling_mask & silencer_modeling_mask.notna()
# Mask to pull out the inactive seqs and the strong enhancers
inactive_modeling_mask = activity_df["group_name_WT"].str.contains("Strong|Inactive")
inactive_modeling_mask = inactive_modeling_mask & inactive_modeling_mask.notna()

# Within the data to model, mask indicating which sequences are strong enhancers
labels_with_silencer = activity_df.loc[silencer_modeling_mask, "group_name_WT"].str.contains("Strong")
labels_with_inactive = activity_df.loc[inactive_modeling_mask, "group_name_WT"].str.contains("Strong")

# Write strong enhancers and silencers to file for the SVM
seq_bins_dir = os.path.join(data_dir, "ActivityBins")
positives_fasta = os.path.join(seq_bins_dir, "strongEnhancer.fasta")
negatives_fasta = os.path.join(seq_bins_dir, "silencer.fasta")
all_strong_mask = activity_df["group_name_WT"].str.contains("Strong")
all_strong_mask = all_strong_mask & all_strong_mask.notna()
strong_ids = activity_df.loc[all_strong_mask, "variant_WT"]
fasta_seq_parse_manip.write_fasta(all_seqs[strong_ids.index + "_" + strong_ids], positives_fasta)
all_silencer_mask = activity_df["group_name_WT"].str.contains("Silencer")
all_silencer_mask = all_silencer_mask & all_silencer_mask.notna()
silencer_ids = activity_df.loc[all_silencer_mask, "variant_WT"]
fasta_seq_parse_manip.write_fasta(all_seqs[silencer_ids.index + "_" + silencer_ids], negatives_fasta)

# Fit k-mer SVM
print("Fitting k-mer Supper Vector Machine. This will take a few minutes.")
# Hyperparameter setup
seed = 1210
word_len = 6
max_mis = 1
nfolds = 5

models_dir = "Models"
svm_dir = os.path.join(models_dir, "StrongEnhancerVsSilencer")
if not os.path.exists(svm_dir):
    os.makedirs(svm_dir)

# Fit the SVM
svm_prefix = os.path.join(svm_dir, f"gkmsvm_{word_len}_{word_len}_{max_mis}")
fig_list, xaxis, svm_tpr, svm_prec, svm_f1, svm_scores = gkmsvm.train_with_cv(positives_fasta, negatives_fasta, svm_prefix, num_folds=nfolds, word_len=word_len, info_pos=word_len, max_mis=max_mis, seed=seed)
plt.close()

# Fit logistic regression models
print("Fitting strong enhancer vs. silencer logistic regression model for CRX occupancy.")
cv = StratifiedKFold(n_splits=nfolds, shuffle=True, random_state=seed)
crx_clf = LogisticRegression()
crx_clf, crx_tpr_list, crx_prec_list, crx_f1_list = modeling.train_estimate_variance(crx_clf, cv, wt_occupancy_df.loc[silencer_modeling_mask, "CRX"], labels_with_silencer, xaxis, positive_cutoff=0)

print("Fitting strong enhancer vs. silencer logistic regression model for 8 TFs.")
occ_clf = LogisticRegression()
param_grid = {"C": np.logspace(-4, 4, 9)}
np.random.seed(seed)
occ_clf, occ_tpr_list, occ_prec_list = modeling.grid_search_hyperparams(occ_clf, nfolds, param_grid, "f1", wt_occupancy_df[silencer_modeling_mask], labels_with_silencer, xaxis, positive_cutoff=0)
c_opt = occ_clf.get_params()["C"]
print(f"Optimal regularization strength (C): {c_opt:1.1e}")

Figure 2.

Strong enhancers contain a diverse array of motifs.

	count	mean	std	min	25%	50%	75%	max
Silencer	837	3.588419	1.848387	0.067069	2.167386	3.408131	4.845272	11.848887
Inactive	928	3.005903	1.690368	0.03447	1.777625	2.810142	3.968906	12.011682
Weak enhancer	1360	3.068334	1.582532	0.010029	1.935493	2.921969	4.031018	12.521734
Strong enhancer	1051	3.782727	1.622289	0.02116	2.577761	3.664645	4.762179	10.185356

(c) Frequency of TF motifs in each activity class.

(d) Frequency of co-occurring TF motifs in strong enhancers. Lower triangle is expected co-occurrence if motifs are independent. (e) Frequency of activity classes, colored as in (b), for sequences in CRX, NRL, and/or MEF2D ChIP-seq peaks. (f) Frequency of TF ChIP-seq peaks in activity classes. TFs in (c) are sorted by feature importance of the logistic regression model in (a).

group_name_WT	Silencer	Inactive	Weak enhancer	Strong enhancer
No binding	0.221493	0.2863	0.331419	0.160788
CRX only	0.203553	0.222276	0.346615	0.227556
CRX+NRL	0.19256	0.115974	0.238512	0.452954
CRX+MEF2D	0.145	0.165	0.28	0.41
All three	0.099338	0.10596	0.284768	0.509934

Lineage-defining TF motifs differentiate strong enhancers from silencers

We performed a de novo motif enrichment analysis to identify motifs that distinguish strong enhancers from silencers and found several differentially enriched motifs matching known TFs. For motifs that matched multiple TFs, we selected one representative TF for downstream analysis, since TFs from the same family have PWMs that are too similar to meaningfully distinguish between motifs for these TFs (Figure 2—figure supplement 2, Materials and methods). Strong enhancers are enriched for several motif families that include TFs that interact with CRX or are important for photoreceptor development: NeuroD1/NDF1 (E-box-binding bHLH) 59Morrow et al.1999, RORB (nuclear receptor) 36Jia et al.200979Srinivas et al.2006, MAZ or Sp4 (C2H2 zinc finger) 51Lerner et al.2005, and NRL (bZIP) 55Mears et al.200156Mitton et al.2000. Sp4 physically interacts with CRX in the retina 51Lerner et al.2005, but we chose to represent the zinc finger motif with MAZ because it has a higher quality score in the HOCOMOCO database 46Kulakovskiy et al.2018. Silencers were enriched for a motif that resembles a partial K50 homeodomain motif but instead matches the zinc finger TF GFI1, a member of the Snail repressor family 8Chiang and Ayyanathan2013 expressed in developing retinal ganglion cells 88Yang et al.2003. Therefore, while strong enhancers and silencers are not distinguished by their CRX motif content, strong enhancers are uniquely enriched for several lineage-defining TFs.

To quantify how well these TF motifs differentiate strong enhancers from silencers, we trained two different classification models with fivefold cross-validation. First, we trained a 6-mer support vector machine (SVM) 19Ghandi et al.2014 and achieved an AUROC of 0.781 ± 0.013 and AUPR of 0.812 ± 0.020 (Figure 2a and Figure 2—figure supplement 1). The SVM considers all 2080 non-redundant 6-mers and provides an upper bound to the predictive power of models that do not consider the exact arrangement or spacing of sequence features. We next trained a logistic regression model on the predicted occupancy for eight lineage-defining TFs (Supplementary file 4) and compared it to the upper bound established by the SVM. In this model, we considered CRX, the five TFs identified in our motif enrichment analysis, and two additional TFs enriched in photoreceptor ATAC-seq peaks 31Hughes et al.2017: RAX, a Q50 homeodomain TF that contrasts with CRX, a K50 homeodomain TF 34Irie et al.2015 and MEF2D, a MADS box TF which co-binds with CRX 2Andzelm et al.2015. The logistic regression model performs nearly as well as the SVM (AUROC 0.698 ± 0.036, AUPR 0.745 ± 0.032, Figure 2a and Figure 2—figure supplement 1) despite a 260-fold reduction from 2080 to 8 features. To determine whether the logistic regression model depends specifically on the eight lineage-defining TFs, we established a null distribution by fitting 100 logistic regression models with randomly selected TFs (Materials and methods). Our logistic regression model outperforms the null distribution (one-tailed Z-test for AUROC and AUPR, p < 0.0008, Figure 2—figure supplement 3), indicating that the performance of the model specifically requires the eight lineage-defining TFs. To determine whether the SVM identified any additional motifs that could be added to the logistic regression model, we generated de novo motifs using the SVM k-mer scores and found no additional motifs predictive of strong enhancers. Finally, we found that our two models perform similarly on an independent test set of CRX-targeted sequences (85White et al.2013; Figure 2—figure supplement 3). Since the logistic regression model performs near the upper bound established by the SVM and depends specifically on the eight selected motifs, we conclude that these motifs comprise nearly all of the sequence features captured by the SVM that distinguish strong enhancers from silencers among CRX-targeted sequences.

Results from de novo motif analysis.

Motifs enriched in strong enhancers (a) and silencers (b). Bottom, de novo motif identified with DREME; top, matched known motif identified with TOMTOM.

Figure 2—figure supplement 3.

Additional validation of the eight transcription factors (TFs) predicted occupancy logistic regression model.

Predictions of the 6-mer support vector machine (SVM) (black) and eight TFs predicted occupancy logistic regression model (orange) on an independent test set. (a) Receiver operating characteristic, (b) precision recall curve. Dashed black line represents chance in both panels.

	CRX	GFI1	MAZ	MEF2D	NDF1	NRL	RORB	RAX
chr1_100559800_SCRUBR	0.274096	2.545296e-13	1.630613e-11	4.707551e-14	1.017487e-7	0.000854	0.00004694361	0.008889
chr1_100559800_UBR	1.178397	5.862032e-11	0.000001102815	1.221394e-10	0.001066875	0.000541	8.777171e-7	0.001608
chr1_100750470_UBR	2.430898	8.232504e-7	5.564299e-11	2.960941e-10	0.01272582	0.969272	0.000001295348	0.001267
chr1_108920170_UBR	2.072197	0.00732386	6.147587e-16	4.758899e-9	2.658399e-10	0.808744	0.005559077	0.003341
chr1_11177090_SCRUBR	3.214338	0.0004034044	4.444271e-14	2.389581e-7	3.62783e-10	0.000005	0.001550753	2.118118

Static version of the figure to display panels (c) and (d). Null distribution of 100 logistic regression models trained using randomly selected motifs (gray) compared to the true features (orange). Shaded area, 1 standard deviation based on fivefold cross-validation. (c) Receiver operating characteristic, (d) precision recall curve. Dashed black line represents chance in both panels.

Figure 2—figure supplement 3c and d, interactive.

# # Read in HOCOMOCO database
# hocomoco = predicted_occupancy.read_pwm_files(os.path.join("Data", "Downloaded", "Pwm", "photoreceptorMotifsAndHOCOMOCOv11_full_MOUSE.meme"))
# hocomoco = hocomoco.apply(predicted_occupancy.ewm_from_letter_prob).apply(predicted_occupancy.ewm_to_dict)

# wt_seqs = all_seqs[all_seqs.index.str.contains("WT")].copy()
# wt_seqs = sequence_annotation_processing.remove_mutations_from_seq_id(wt_seqs)
# wt_seqs = wt_seqs[activity_df.index]
# modeling_seqs = wt_seqs[silencer_modeling_mask]

# niter = 100
# nfeatures = len(ewms)
# # Track the cross-validated mean TPR and precision for each feature set
# random_tprs = []
# random_precs = []
# # Keep track of the features selected for each round
# random_ewms = []

# np.random.seed(seed)
# for i in range(niter):
#     if i % 10 == 9:
#         print(f"Iteration {i+1}")
        
#     # Randomly sample PWMs
#     sample = hocomoco.sample(nfeatures)
#     random_ewms.append(sample.index.str.split("_").str[0].values)
#     # Do predicted occupancy scan
#     features = predicted_occupancy.all_seq_total_occupancy(modeling_seqs, sample, mu, convert_ewm=False)
#     # Fit the model
#     clf = LogisticRegression(C=c_opt)
#     clf, tpr, prec, f1 = modeling.train_estimate_variance(clf, cv, features, labels_with_silencer, xaxis, positive_cutoff=0)
    
#     # Store the result
#     random_tprs.append(np.mean(tpr, axis=0))
#     random_precs.append(np.mean(prec, axis=0))
    
# fig, ax_list = plot_utils.setup_multiplot(2, n_cols=2, sharex=False, sharey=False)
# niter_rand = len(random_occ_tprs)
# rand_tpr_plotting = [[j] for i, j in random_occ_tprs.iterrows()] + [occ_tpr_cv]
# rand_prec_plotting = [[j] for i, j in random_occ_precs.iterrows()] + [occ_prec_cv]
# rand_names = [""]  * niter_rand + ["True features"]
# rand_colors = ["#8080801A"] * niter_rand + ["#E69B04"]

# _, background_aurocs, _, background_auprs, _ = plot_utils.roc_pr_curves(
#     modeling_xaxis, rand_tpr_plotting, rand_prec_plotting, rand_names, model_colors=rand_colors,
#     prc_chance=prc_chance, figax=([fig, fig], ax_list)
# )

# plot_utils.add_letter(ax_list[0], -0.15, 1.03, "c")
# plot_utils.add_letter(ax_list[1], -0.15, 1.03, "d")

# # KS test, null hypothesis: random AUROCs and AUPRs are normally distributed
# # One-tailed Z-test that the real data is drawn from this distribution
# for data, name in zip([background_aurocs, background_auprs], ["AUROC", "AUPR"]):
#     real, rand = data[niter_rand], data[:niter_rand]
#     dstat, pval = stats.kstest(stats.zscore(rand), "norm")
#     print(f"{name}s of random features are normally distributed, KS test p = {pval:.2f}, D = {dstat:.2f}")
#     zscore = (real - np.mean(rand)) / np.std(rand)
#     pval = stats.norm.cdf(-np.abs(zscore))
#     print(f"Probability that the {name} of the real features is drawn from the background distribution, one-tailed Z-test p = {pval:2f}")

# display(fig)
# plt.close()

Interactive version of panels (c) and (d). Note that this takes close to an hour to run.

Strong enhancers are characterized by diverse total motif content

To understand how these eight TF motifs differentiate strong enhancers from silencers, we first calculated the total predicted occupancy of each sequence by all eight lineage-defining TFs and compared the different activity classes. Strong enhancers and silencers both have higher total predicted occupancies than inactive sequences, but total predicted occupancies do not distinguish strong enhancers and silencers from each other (Figure 2b, Supplementary file 5). Since strong enhancers are enriched for several motifs relative to silencers, this suggests that strong enhancers are distinguished from silencers by the diversity of their motifs, rather than the total number.

We considered two hypotheses for how the more diverse collection of motifs function in strong enhancers: either strong enhancers depend on specific combinations of TF motifs (‘TF identity hypothesis’) or they instead must be co-occupied by multiple lineage-defining TFs, regardless of TF identity (‘TF diversity hypothesis’). To distinguish between these hypotheses, we examined which specific motifs contribute to the total motif content of strong enhancers and silencers. We considered motifs for a TF present in a sequence if the TF predicted occupancy was above 0.5 molecules (Supplementary file 4), which generally corresponds to at least one motif with a relative K_D above 3%. This threshold captures the effect of low affinity motifs that are often biologically relevant 10Crocker et al.201515Farley et al.201516Farley et al.201663Parker et al.2011. As expected, 97% of strong enhancers and silencers contain CRX motifs since the sequences were selected based on CRX binding or significant matches to the CRX PWM within open chromatin (Figure 2c). Compared to silencers, strong enhancers contain a broader diversity of motifs for the eight lineage-defining TFs (Figure 2c). However, while strong enhancers contain a broader range of motifs, no single motif occurs in a majority of strong enhancers: NRL motifs are present in 23% of strong enhancers, NeuroD1 and RORB in 18% each, and MAZ in 16%. Additionally, none of the motifs tend to co-occur as pairs in strong enhancers: no specific pair occurred in more than 5% of sequences (Figure 2d). We also did not observe a bias in the linear arrangement of motifs in strong enhancers (Materials and methods). Similarly, no single motif occurs in more than 15% of silencers (Figure 2c). These results suggest that strong enhancers are defined by the diversity of their motifs, and not by specific motif combinations or their linear arrangement.

The results above predict that strong enhancers are more likely to be bound by a diverse but degenerate collection of TFs, compared with silencers or inactive sequences. We tested this prediction by examining in vivo TF binding using published ChIP-seq data for NRL 23Hao et al.2012 and MEF2D 2Andzelm et al.2015. Consistent with the prediction, sequences bound by CRX and either NRL or MEF2D are approximately twice as likely to be strong enhancers compared to sequences only bound by CRX (Figure 2e). Sequences bound by all three TFs are the most likely to be strong or weak enhancers rather than silencers or inactive sequences. However, most strong enhancers are not bound by either NRL or MEF2D (Figure 2f), indicating that binding of these TFs is not required for strong enhancers. Our results support the TF diversity hypothesis: CRX-targeted enhancers are co-occupied by multiple TFs, without a requirement for specific combinations of lineage-defining TFs.

Strong enhancers have higher motif information content than silencers

Our results indicate that both strong enhancers and silencers have a higher total motif content than inactive sequences, while strong enhancers contain a more diverse collection of motifs than silencers. To quantify these differences in the number and diversity of motifs, we computed the information content of CRX-targeted sequences using Boltzmann entropy. The Boltzmann entropy of a system is related to the number of ways the system’s molecules can be arranged, which increases with either the number or diversity of molecules (67Phillips et al.2012, Chapter 5). In our case, each TF is a different type of molecule and the number of each TF is represented by its predicted occupancy for a cis-regulatory sequence. The number of molecular arrangements is thus W, the number of distinguishable permutations that the TFs can be ordered on the sequence, and the information content of a sequence is then log₂W (Materials and methods).

We found that on average, strong enhancers have higher information content than both silencers and inactive sequences (Mann-Whitney U test, p = 1 × 10^–23 and 7 × 10^–34, respectively, Figure 3a, Supplementary file 5), confirming that information content captures the effect of both the number and diversity of motifs. Quantitatively, the average silencer and inactive sequence contains 1.6 and 1.4 bits, respectively, which represents approximately three total motifs for two TFs. Strong enhancers contain on average 2.4 bits, representing approximately three total motifs for three TFs or four total motifs for two TFs. To compare the predictive value of our information content metric to the model based on all eight motifs, we trained a logistic regression model and found that information content classifies strong enhancers from silencers with an AUROC of 0.634 ± 0.008 and an AUPR of 0.663 ± 0.014 (Figure 3b and Figure 3—figure supplement 1). This is only slightly worse than the model trained on eight TF occupancies despite an eightfold reduction in the number of features, which is itself comparable to the SVM with 2080 features. The difference between the two logistic regression models suggests that the specific identities of TF motifs make some contribution to the eight TF model, but that most of the signal captured by the SVM can be described with a single metric that does not assign weights to specific motifs. Information content also distinguishes strong enhancers from inactive sequences (AUROC 0.658 ± 0.012, AUPR 0.675 ± 0.019, Figure 3b and Figure 3—figure supplement 1). These results indicate that strong enhancers are characterized by higher information content, which reflects both the total number and diversity of motifs.

Figure 3: Information content classifies strong enhancers.

(a) Information content for different activity classes. (b) Receiver operating characteristic of information content to classify strong enhancers from silencers (orange) or inactive sequences (indigo).

Figure 3—figure supplement 1: Precision recall curve of logistic regression classifier using information content.

Orange, strong enhancer vs. silencer; indigo, strong enhancer vs. inactive; shaded area, 1 standard deviation based on fivefold cross-validation.

	count	mean	std	min	25%	50%	75%	max
Silencer	837	1.554721	1.872824	0.000173	0.195721	0.952877	2.240308	15.248629
Inactive	928	1.385812	1.646322	0.000105	0.150796	0.841681	2.050814	14.738741
Weak enhancer	1360	1.49678	1.683849	0.000008	0.201747	1.014613	2.216628	17.960698
Strong enhancer	1051	2.383258	2.1786	0.000173	0.635291	1.836731	3.453384	13.082139

Strong enhancers require high information content but not NRL motifs

Our results show that except for CRX, none of the lineage-defining motifs occur in a majority of strong enhancers. However, all sequences were tested in reporter constructs with the Rho promoter, which contains an NRL motif and three CRX motifs 9Corbo et al.201047Kwasnieski et al.2012. Since NRL is a key co-regulator with CRX in rod photoreceptors, we tested whether strong enhancers generally require NRL, which would be inconsistent with our TF diversity hypothesis. We removed the NRL motif by recloning our MPRA library without the basal Rho promoter. If strong enhancers require an NRL motif for high activity, then only CRX-targeted sequences with NRL motifs will drive reporter expression. If information content (i.e. total motif content and diversity) is the primary determinant of strong enhancers, only CRX-targeted sequences with sufficient motif diversity, measured by information content, will drive reporter expression regardless of whether or not NRL motifs are present.

We replaced the Rho promoter with a minimal 23 bp polylinker sequence between our libraries and DsRed, and repeated the MPRA (Figure 1—figure supplement 1, Supplementary file 3). CRX-targeted sequences were designated as ‘autonomous’ if they retained activity in the absence of the Rho promoter (log₂(RNA/DNA) > 0, Materials and methods). We found that 90% of autonomous sequences are from the enhancer class, while less than 3% of autonomous sequences are from the silencer class (Figure 4a). This confirms that the distinction between silencers and enhancers does not depend on the Rho promoter, which is consistent with our previous finding that CRX-targeted silencers repress other promoters 32Hughes et al.201886White et al.2016. However, while most autonomous sequences are enhancers, only 39% of strong enhancers and 9% of weak enhancers act autonomously. Consistent with a role for information content, autonomous strong enhancers have higher information content (Mann-Whitney U test p = 4 × 10^–8, Figure 4b) and higher predicted CRX occupancy (Mann-Whitney U test p = 9 × 10^–12, Figure 4c) than non-autonomous strong enhancers. We found no evidence that specific lineage-defining motifs are required for autonomous activity, including NRL, which is present in only 25% of autonomous strong enhancers (Figure 4d). Similarly, NRL ChIP-seq binding 23Hao et al.2012 occurs more often among autonomous strong enhancers (41% vs. 19%, Fisher’s exact test p = 2 × 10^–14, odds ratio = 3.0), yet NRL binding still only accounts for a minority of these sequences. We thus conclude that strong enhancers require high information content, rather than any specific lineage-defining motifs.

Sequence features of autonomous and non-autonomous strong enhancers.

(a) Activity of library in the presence (x-axis) or absence (y-axis) of the *Rho* promoter. Dark blue, strong enhancers; light blue, weak enhancers; green, inactive; red, silencers; gray, ambiguous; horizontal line, cutoff for autonomous activity. Points on the far left and/or very bottom are sequences that were present in the plasmid pool but not detected in the RNA. (**b–d**) Comparison of autonomous and non-autonomous strong enhancers for information content (b), predicted cone-rod homeobox (CRX) occupancy (c), and frequency of transcription factor (TF) motifs (d).

	count	mean	std	min	25%	50%	75%	max
False	635	2.073301	1.96416	0.000173	0.488725	1.624789	3.026204	11.747577
True	402	2.888074	2.424544	0.000346	0.990757	2.272392	4.401275	13.082139

	count	mean	std	min	25%	50%	75%	max
False	635	2.34943	1.154518	0.003694	1.471752	2.255551	3.075332	7.3685
True	402	2.83343	1.127028	0.015596	2.062315	2.858271	3.554521	5.852791

nrl_bound	False	True
False	513	122
True	236	166

TF motifs contribute independently to strong enhancers

Our results indicate that information content distinguishes strong enhancers from silencers and inactive sequences. Information content only takes into account the total number and diversity of motifs in a sequence and not any potential interactions between them. The classification success of information content thus suggests that each TF motif will contribute independently to enhancer activity. We tested this prediction with CRX-targeted sequences where all CRX motifs were abolished by point mutation (Supplementary file 3). Consistent with our previous work 85White et al.2013, mutating CRX motifs causes the activities of both enhancers and silencers to regress toward basal levels (Pearson’s r = 0.608, Figure 5a), indicating that most enhancers and silencers show some dependence on CRX. However, 40% of wild-type strong enhancers show low CRX dependence and remain strong enhancers with their CRX motifs abolished. Although strong enhancers with high and low CRX dependence have similar wild-type information content (Figure 5b), strong enhancers with low CRX dependence have lower predicted CRX occupancy than those with high CRX dependence (Mann-Whitney U test p = 2 × 10^–9, Figure 5c), and also have higher ‘residual’ information content (i.e. information content without CRX motifs, Mann-Whitney U test p = 1 × 10^–7, Figure 5d). Low CRX dependence sequences have an average of 1.5 residual bits, which corresponds to three motifs for two TFs, while high CRX dependence sequences have an average of 1.0 residual bits, which corresponds to two motifs for two TFs (Figure 5e).

Independence of transcription factor (TF) motifs in strong enhancers.

(a) Activity of sequences with and without cone-rod homeobox (CRX) motifs. Points are colored by the activity group with CRX motifs intact: dark blue, strong enhancers; light blue, weak enhancers; green, inactive; red, silencers; gray, ambiguous; horizontal dotted lines and color bar represent the cutoffs for the same groups when CRX motifs are mutated. Solid black line is the y = x line. (**b–d**) Comparison of strong enhancers with high and low CRX dependence for information content (b), predicted CRX occupancy (c), and residual information content (d). (e) Representative strong enhancers with high (top) or low (bottom) CRX dependence.

	count	mean	std	min	25%	50%	75%	max
False	586	2.321663	2.067846	0.000346	0.64176	1.849581	3.333561	11.676515
True	344	2.857066	2.411316	0.001591	1.145032	2.413095	3.969305	13.082139

	count	mean	std	min	25%	50%	75%	max
False	586	2.87682	1.069855	0.927761	2.085474	2.857976	3.57521	7.3685
True	344	2.454016	1.009155	0.964684	1.626097	2.338718	3.110073	5.730406

	count	mean	std	min	25%	50%	75%	max
False	586	1.026129	1.283253	0.000001	0.09754	0.493644	1.472322	7.129248
True	344	1.536551	1.638575	0.000136	0.264872	1.046836	2.33869	9.819172

Strong enhancers with low and high CRX dependence have similar wild-type information content and similar total predicted occupancy (Figure 5b and e). As a result, sequences with more CRX motifs have fewer motifs for other TFs, suggesting that there is no evolutionary pressure for enhancers to contain additional motifs beyond the minimum amount of information content required to be active. To test this idea, we calculated the minimum number and diversity of motifs necessary to specify a relatively unique location in the genome 87Wunderlich and Mirny2009 and found that a 164 bp sequence only requires five motifs for three TFs (Materials and methods). These motif requirements can be achieved in two ways with similar information content that differ only in the quantitative number of motifs for each TF. In other words, the number of motifs for any particular TF is not important so long as there is sufficient information content. Taken together, we conclude that each TF motif provides an independent contribution toward specifying strong enhancers.

Discussion

Many regions in the genome are bound by TFs and bear the epigenetic hallmarks of active cis-regulatory sequences, yet fail to exhibit cis-regulatory activity when tested directly. The discrepancy between measured epigenomic state and cis-regulatory activity indicates that enhancers and silencers consist of more than the minimal sequence features necessary to recruit TFs and chromatin-modifying factors. Our results show that enhancers, silencers, and inactive sequences in developing photoreceptors can be distinguished by their motif content, even though they are indistinguishable by CRX binding or chromatin accessibility. We show that both enhancers and silencers contain more TF motifs than inactive sequences, and that enhancers also contain more diverse sets of motifs for lineage-defining TFs. These differences are captured by our measure of information content. Information content, as a single metric, identifies strong enhancers nearly as well as an unbiased set of 2080 non-redundant 6-mers used for an SVM, indicating that a simple measure of motif number and diversity can capture the key sequence features that distinguish enhancers from other sequences that lie in open chromatin.

The results of our information content classifier are consistent with the TF collective model of enhancers 39Junion et al.201278Spitz and Furlong2012: globally, active enhancers are specified by the combinatorial action of lineage-defining TFs with little constraint on which motifs must co-occur. We show that CRX-targeted enhancers are distinguished from inactive CRX-targeted sequences by a larger, more diverse collection of TF motifs, and not any specific combination of motifs. This indicates that enhancers are active because they have acquired the necessary number of TF binding motifs, and not because they are defined by a strict regulatory grammar. Sequences with fewer motifs may be bound by CRX and reside within open chromatin, but they lack sufficient TF binding for activity. Such loose constraints would facilitate the de novo emergence of tissue-specific enhancers and silencers over evolution and explain why critical cell type-specific TF interactions, such as CRX and NRL in rod photoreceptors, occur at only a minority of the active enhancers in that cell type 28Hsiau et al.200732Hughes et al.201885White et al.2013.

Like enhancers, CRX-targeted silencers require higher motif content and are dependent on CRX motifs, but they lack the TF diversity of enhancers. The lack of TF diversity in silencers parallels the architecture of signal-responsive cis-regulatory sequences, which are silencers in the absence of a signal and require multiple activators for induction 4Barolo and Posakony2002. Consistent with this, we previously showed using synthetic sequences that high occupancy of CRX alone is sufficient to encode silencers while the addition of a single NRL motif converts synthetic silencers to enhancers, and that genomic sequences with very high CRX motif content repress a basal promoter that lacks NRL motifs 86White et al.2016. We found that photoreceptor genes which are de-repressed upon loss of CRX are located near cis-regulatory sequences with high CRX motif content, and that genes near regions that are bound only by CRX are expressed at lower levels than genes near regions co-bound by CRX and NRL 86White et al.2016. In the current study, we find that silencers in our MPRA library are more likely to occur near de-repressed photoreceptor genes, while strong enhancers are enriched near genes that lose expression in Crx^-/- retina. These findings suggest that the low TF diversity and high CRX motif content that characterize silencers in our MPRA library are also important for silencing in the genome.

The contrast in motif diversity between enhancers and silencers that we observe could explain how CRX achieves selective activation and repression of its target genes in multiple cell types and across developmental time points 60Murphy et al.201972Ruzycki et al.2018. CRX itself is required for silencing, and we previously showed that some silencers become active enhancers in Crx^-/- retina 86White et al.2016. The mechanism of CRX-based silencing is unknown, however CRX cooperates with other TFs that can sometimes act as repressors of cell type-specific genes 7Chen et al.200565Peng et al.200584Webber et al.2008, while other repressors can directly inhibit activation by CRX or its co-activators 12Dorval et al.200626Hlawatsch et al.201357Mitton et al.200375Sanuki et al.2010. In Drosophila photoreceptors, selective silencing of opsin genes is controlled by cell type-specific expression of a repressor, Dve, which acts on the same K50 homeodomain-binding sites as a universally expressed activator, Otd, a homolog of CRX 70Rister et al.2015. Other transcriptional activators selectively act as repressors in the same cell type. GATA-1 represses the _GATA-_2 promoter by displacing CREB-binding protein (CBP), while at other genes GATA-1 binds CBP to activate transcription 21Grass et al.2003. Selective repression by GATA-1 is also mediated by chromatin occupancy levels and interaction with co-regulators 38Johnson et al.2006, which is consistent with our finding that sequence context enables a TF to both activate and repress genes in the same cell type.

Given the central role of CRX in selectively regulating genes in multiple closely related cell types 60Murphy et al.2019, we speculate that CRX-targeted silencers may contain sufficient information to act as enhancers in other cell types in which a different set of co-activating TFs are expressed. This hypothesis would be consistent with the finding that many silencers are enhancers in other cell types 11Doni Jayavelu et al.202020Gisselbrecht et al.202061Ngan et al.2020. Our work suggests that characterizing sequences by their motif information content offers a way to identify these different classes of cis-regulatory sequences in the genome.

Materials and methods

Key resources table
Reagent type (species) or resource	Designation	Source or reference	Identifiers	Additional information
Strain, strain background (Mus musculus, male and female)	CD-1	Charles River	Strain code 022
Recombinant DNA reagent	Library1	This paper		Listed in Supplementary file 1
Recombinant DNA reagent	Library2	This paper		Listed in Supplementary file 2
Recombinant DNA reagent	pJK01_Rhominprox-DsRed	47Kwasnieski et al.2012	AddGene plasmid # 173,489
Recombinant DNA reagent	pJK03__Rho_basal__DsRed	47Kwasnieski et al.2012	AddGene plasmid # 173,490
Sequence-based reagent	Primers	IDT		Listed in Supplementary file 6
Commercial assay or kit	Monarch PCR Cleanup Kit	New England Biolabs	T1030S
Commercial assay or kit	Monarch DNA Gel Extraction Kit	New England Biolabs	T1020L
Commercial assay or kit	TURBO DNA-free	Invitrogen	AM1907
Commercial assay or kit	SuperScript III Reverse Transcriptase	Invitrogen	18080044
Software, algorithm	Bedtools	https://bedtools.readthedocs.io/en/latest/	RRID:SCR_006646
Software, algorithm	MEME Suite	https://meme-suite.org/	RRID:SCR_001783
Software, algorithm	ShapeMF	https://github.com/h-samee/shape-motif, 74Samee2021	DOI:10.1016/j.cels.2018.12.001
Software, algorithm	Numpy	https://numpy.org/	DOI:10.1038/s41586-020-2649-2
Software, algorithm	Scipy	https://www.scipy.org/	DOI:10.1038/s41592-019-0686-2
Software, algorithm	Pandas	https://pandas.pydata.org/	DOI:10.5281/zenodo.3509134
Software, algorithm	Matplotlib	https://matplotlib.org/	DOI:10.5281/zenodo.1482099
Software, algorithm	Logomaker	https://github.com/jbkinney/logomaker, 40Justin2021	DOI:10.1093/bioinformatics/btz921

Library design

CRX ChIP-seq peaks re-processed by 72Ruzycki et al.2018 were intersected with previously published CRX MPRA libraries 32Hughes et al.201885White et al.2013 and one unpublished library to select sequences that had not been previously tested by MPRA. These sequences were scanned for instances of CRX motifs using FIMO version 4.11.2 3Bailey et al.2009, a p-value cutoff of 2.3 × 10^–3 (see below), and a CRX PWM derived from an electrophoretic mobility shift assay 49Lee et al.2010. We centered 2622 sequences on the highest scoring CRX motif. For 677 sequences without a CRX motif, we instead centered them using the Gibbs sampler from ShapeMF (Github commit abe8421) 73Samee et al.2019 and a motif size of 10.

For sequences unbound in CRX ChIP-seq but in open chromatin, we took ATAC-seq peaks collected in 8-week FACS-purified rods, green cones, and Nrl^-/- blue cones 31Hughes et al.2017 and removed sequences that overlapped with CRX ChIP-seq peaks. The remaining sequences were scanned for instances of CRX motifs using FIMO with a p-value cutoff of 2.5 × 10^–3 and the CRX PWM. Sequences with a CRX motif were kept and the three ATAC-seq data sets were merged together, intersected with H3K27ac and H3K4me3 ChIP-seq peaks collected in P14 retinas 72Ruzycki et al.2018, and centered on the highest scoring CRX motifs. We randomly selected 1004 H3K27ac⁺H3K4me3^- sequences and 541 H3K27ac⁺H3K4me3⁺ to reflect the fact that ~35% of CRX ChIP-seq peaks are H3K4me3⁺. After synthesis of our library, we discovered 11% of these sequences do not actually overlap H3K27ac ChIP-seq peaks (110/1004 of the H3K4me3^- group and 60/541 of the H3K4me3⁺ group), but we still included them in the analysis because they contain CRX motifs in ATAC-seq peaks.

All data was converted to mm10 coordinates using the UCSC liftOver tool 22Haeussler et al.2019 and processed using Bedtools version 2.27.1 68Quinlan and Hall2010. All sequences in our library design were adjusted to 164 bp and screened for instances of EcoRI, SpeI, SphI, and NotI sites. In total, our library contains 4844 genomic sequences (2622 CRX ChIP-seq peaks with motifs, 677 CRX ChIP-seq peaks without motifs, 1004 CRX^-ATAC⁺H3K27ac⁺H3K4me3^- CRX motifs, and 541 CRX^-ATAC⁺H3K27ac⁺H3K4me3⁺ CRX motifs), a variant of each sequence with all CRX motifs mutated, 150 scrambled sequences, and a construct for cloning the basal promoter alone.

For sequences centered on CRX motifs, all CRX motifs with a p-value of 2.5 × 10^–3 or less were mutated by changing the core TAAT to TACT 49Lee et al.2010 on the appropriate strand, as described previously 32Hughes et al.201885White et al.2013. We then re-scanned sequences and mutated any additional motifs inadvertently created.

To generate scrambled sequences, we randomly selected 150 CRX ChIP-seq peaks spanning the entire range of GC content in the library. We then scrambled each sequence while preserving dinucleotide content as previously described 85White et al.2013. We used FIMO to confirm that none of the scrambled sequences contain CRX motifs.

We unintentionally used a FIMO p-value cutoff of 2.3 × 10^–3 to identify CRX motifs in CRX ChIP-seq peaks, rather than the slightly less stringent 2.5 × 10^–3 cutoff used with ATAC-seq peaks or mutating CRX motifs. Due to this anomaly, there may be sequences centered using ShapeMF that should have been centered on a CRX motif, and these motifs would not have been mutated because CRX motifs were not mutated in sequences centered using ShapeMF. However, any intact CRX motifs would still be captured in the residual information content of the mutant sequence.

Plasmid library construction

We generated two 15,000 libraries of 230 bp oligonucleotides (oligos) from Agilent Technologies (Santa Clara, CA) through a limited licensing agreement. Our library was split across the two oligo pools, ensuring that both the genomic and mutant forms of each sequence were placed in the same oligo pool (Supplementary files 1 and 2). Both oligo pools contain all 150 scrambled sequences as an internal control. All sequences were assigned three unique barcodes as previously described 85White et al.2013. In each oligo pool, the basal promoter alone was assigned 18 unique barcodes. Oligos were synthesized as follows: 5’ priming sequence (GTAGCGTCTGTCCGT)/EcoRI site/Library sequence/SpeI site/C/SphI site/Barcode sequence/NotI site/3’ priming sequence (CAACTACTACTACAG). To clone the basal promoter into barcoded oligos without any upstream cis-regulatory sequence, we placed the SpeI site next to the EcoRI site, which allowed us to place the promoter between the EcoRI site and the 3’ barcode.

We cloned the synthesized oligos as previously described by our group 47Kwasnieski et al.201286White et al.201685White et al.2013. Specifically, for each oligo pool, we used 50 femtomoles of template and four cycles of PCR in each of multiple 50 µl reactions (New England Biolabs [NEB], Ipswich, MA) (NEB Phusion) using primers MO563 and MO564 (Supplementary file 6), 2% DMSO, and an annealing temperature of 57°C. PCR amplicons were purified from a 2% agarose gel (NEB), digested with EcoRI-HF and NotI-HF (NEB), and then cloned into the EagI and EcoRI sites of the plasmid pJK03 with multiple 20 µl ligation reactions (NEB T4 ligase). The libraries were transformed into 5-alpha electrocompetent cells (NEB) and grown in liquid culture. Next, 2 µg of each library was digested with SphI-HF and SpeI-HF (NEB) and then treated with Antarctic phosphatase (NEB).

The Rho basal promoter and DsRed reporter gene was amplified from the plasmid pJK01 using primers MO566 and MO567 (Supplementary file 6). The Polylinker and DsRed reporter gene was amplified from the plasmid pJK03 using primers MO610 and MO567 (Supplementary file 6). The Polylinker is a short 23 bp multiple cloning site with no known core promoter motifs. Inserts were purified from a 1% agarose gel (NEB), digested with NheI-HF and SphI-HF (NEB), and cloned into the libraries using multiple 20 µl ligations (NEB T4 ligase). The libraries were transformed into 5-alpha electrocompetent cells (NEB) and grown in liquid culture.

Retinal explant electroporation

Animal procedures were performed in accordance with a Washington University in St Louis Institutional Animal Care and Use Committee-approved vertebrate animals protocol. Electroporation into retinal explants and RNA extraction was performed as described previously 28Hsiau et al.200732Hughes et al.201847Kwasnieski et al.201286White et al.201685White et al.2013. Briefly, retinas were isolated from P0 newborn CD-1 mice and electroporated in a solution with 30 µg library and 30 µg Rho-GFP. Electroporated retinas were cultured for 8 days, at which point they were harvested, washed three times with HBSS (ThermoFisher Scientific/Gibco, Waltham, MA), and stored in TRIzol (ThermoFisher Scientific/Invitrogen, Waltham, MA) at –80°C. Five retinas were pooled for each biological replicate and three replicates were performed for each library. RNA was extracted from TRIzol according to manufacturer’s instructions and treated with TURBO DNase (Invitrogen). cDNA was prepared using SuperScript RT III (Invitrogen) with oligo dT primers. Barcodes from both the cDNA and the plasmid DNA pool were amplified for sequencing (described below). The resulting products were mixed at equal concentration and sequenced on the Illumina NextSeq platform. We obtained greater than 1300× coverage across all samples.

Rho libraries were amplified using primers MO574 and MO575 (Supplementary file 6) for six cycles at an annealing temperature of 66°C followed by 18 cycles with no annealing step (NEB Phusion) and then purified with the Monarch PCR kit (NEB). PCR amplicons were digested using MfeI-HF and SphI-HF (NEB) and ligated to custom-annealed adaptors with PE2 indexing barcodes and phased P1 barcodes (Supplementary file 6). The final enrichment PCR used primers MO588 and MO589 (Supplementary file 6) for 20 cycles at an annealing temperature of 66°C (NEB Phusion), followed by purification with the Monarch PCR kit. Polylinker libraries were amplified using primers BC_CRX_Nested_F and BC_CRX_R (Supplementary file 6) for 30 cycles (NEB Q5) at an annealing temperature of 67°C and then purified with the Monarch PCR kit. Illumina adaptors were then added via two further rounds of PCR. First, P1 indexing barcodes were added using forward primers P1_inner_A through P1_inner_D and reverse primer P1_inner_nested_rev (Supplementary file 6) for five cycles at an annealing temperature of 55°C followed by five cycles with no annealing step (NEB Q5). PE2 indexing barcodes were then added by amplifying 2 µl of the previous reaction with forward primer P1_outer and reverse primers PE2_outer_SIC69 and PE2_outer_SIC70 (Supplementary file 6) for five cycles at an annealing temperature of 66°C followed by five cycles with no annealing step (NEB Q5) and then purified with the Monarch PCR kit.

Data processing

All data processing, statistical analysis, and downstream analyses were performed in Python version 3.6.5 using Numpy version 1.15.4 24Harris et al.2020, Scipy version 1.1.0 82Virtanen et al.2020, and Pandas version 0.23.4 54McKinney2010, and visualized using Matplotlib version 3.0.2 33Hunter2007 and Logomaker version 0.8 81Tareen and Kinney2020. All statistical analysis used two-sided tests unless noted otherwise.

Sequencing reads were filtered to ensure that the barcode sequence perfectly matched the expected sequence (>93% reads in a sample for the Rho libraries, >86% reads for the Polylinker libraries). For the Rho libraries, barcodes that had less than 10 raw counts in the DNA sample were considered missing and removed from downstream analysis. Barcodes that had less than five raw counts in any cDNA sample were considered present in the input plasmid pool but below the detection limit and thus set to zero in all samples. Barcode counts were normalized by reads per million (RPM) for each sample. Barcode expression was calculated by dividing the cDNA RPM by the DNA RPM. Replicate-specific expression was calculated by averaging the barcodes corresponding to each library sequence. After performing statistical analysis (see below), expression levels were normalized by replicate-specific basal mean expression and then averaged across biological replicates.

For the Polylinker assay, the expected lack of expression of many constructs required different processing. Barcodes that had less than 50 raw counts in the DNA sample were removed from downstream analysis. Barcodes were normalized by RPM for each replicate. Barcodes that had less than 8 RPM in any cDNA sample were set to zero in all samples. cDNA RPM were then divided by DNA RPM as above. Within each biological replicate, barcodes were averaged as above but were not normalized to basal expression because there is no basal construct. Expression values were then averaged across biological replicates. Due to the low expression of scrambled sequences and the lack of a basal construct, we were unable to assess data calibration with the same rigor as above.

Assignment of activity classes

Activity classes were assigned by comparing expression levels to basal promoter expression levels across replicates. The null hypothesis is that the expression of a sequence is the same as basal levels. Expression levels were approximately log-normally distributed, so we computed the log-normal parameters for each sequence and then performed Welch’s t-test. We corrected for multiple hypotheses using the Benjamini-Hochberg FDR procedure. We corrected for multiple hypotheses in each library separately to account for any potential batch effects between libraries. The log₂ expression was calculated after adding a pseudocount of 1 × 10^–3 to every sequence.

Sequences were classified as enhancers if they were twofold above basal and the q-value was below 0.05. Silencers were similarly defined as twofold below basal and q-value less than 0.05. Inactive sequences were defined as within a twofold change and q-value greater than or equal to 0.05. All other sequences were classified as ambiguous and removed from further analysis. We used scrambled sequences to further stratify enhancers into strong and weak enhancers, using the rationale that scrambled sequences give an empirical distribution for the activity of random sequences. We defined strong enhancers as enhancers that are above the 95th percentile of scrambled sequences and all other enhancers as weak enhancers.

For the Polylinker assay, we did not have a basal construct as a reference point. Instead, we defined a sequence to have autonomous activity if the average cDNA barcode counts were higher than average DNA barcode counts, and non-autonomous otherwise. The log₂ expression was calculated after adding a pseudocount of 1 × 10^–2 to every sequence.

RNA-seq analysis

We obtained RNA-seq data from WT and Crx^-/- P21 retinas 71Roger et al.2014 processed into a counts matrix for each gene by 72Ruzycki et al.2018. Each sample was normalized by read counts per million and replicates were averaged together. Genes with at least a twofold change between genotypes were considered differentially expressed. We determined which differentially expressed genes are near a member of our library using previously published associations between retinal ATAC-seq peaks and genes 60Murphy et al.2019. For de-repressed genes, we determined how often the nearest library member is a silencer; for down-regulated genes, we determined how often the nearest library member is a strong or weak enhancer.

Motif analysis

We performed motif enrichment analysis using the MEME Suite version 5.0.4 3Bailey et al.2009. We searched for motifs that were enriched in one group of sequences relative to another group using DREME-py3 with the parameters -mink 6 -maxk 12 -e 0.05 and compared the de novo motifs to known motifs using TOMTOM on default parameters. We ran DREME using strong enhancers as positives and silencers as negatives, and vice versa. For TOMTOM, we used version 11 of the full mouse HOCOMOCO database 46Kulakovskiy et al.2018 with the following additions from the JASPAR human database 42Khan et al.2018: NRL (accession MA0842.1), RORB (accession MA1150.1), and RAX (accession MA0718.1). We added these PWMs because they have known roles in the retina, but the mouse PWMs were not in the HOCOMOCO database. We also used the CRX PWM that we used to design the library. Motifs were selected for downstream analysis based on their matches to the de novo motifs, whether the TF had a known role in retinal development, and the quality of the PWM. Because PWMs from TFs of the same family were so similar, we used one TF for each DREME motif, recognizing that these motifs may be bound by other TFs that recognize similar motifs. We did not use any PWMs with a quality of ‘D’. We excluded DREME motifs without a match to the database from further analysis; most of these resemble dinucleotides.

Predicted occupancy

We computed predicted occupancy as previously described 85White et al.201389Zhao et al.2009. Briefly, we normalized each letter probability matrix by the most probable letter at each position. We took the negative log of this matrix and multiplied by 2.5, which corresponds to the ideal gas constant times 300 K, to obtain an energy weight matrix. We used a chemical potential μ of 9 for all TFs. At this value, the probability of a site being bound is at least 0.5 if the relative K_D is at least 0.03 of the optimal binding site. We computed the predicted occupancy for every site on the forward and reverse strands and summed them together to get a single value for each TF.

To determine if there is a bias in the linear arrangement of motifs, we selected strong enhancers with exactly one site occupied by CRX and exactly one site occupied by a second TF. We counted the number of times the position of the second TF was 5’ and 3’ of the CRX site and then performed a binomial test. We did not observe a statistically significant bias for any TF at an FDR q-value cutoff of 0.05. We also performed this analysis for silencers with exactly one site occupied by CRX and exactly one site occupied by NRL and did not observe a significant difference in the 5’ vs. 3’ bias of strong enhancers vs. silencers (Fisher’s exact test p = 0.17).

Information content

To capture the effects of TF predicted occupancy and diversity in a single metric, we calculated the motif information content using Boltzmann entropy. Boltzmann’s equation states that the entropy of a system $S$ is related to the number of ways the molecules can be arranged (microstates) $W$ via the equation $S = k_{B} l o g W$ , where $k_{B}$ is Boltzmann’s constant (67Phillips et al.2012, Chapter 5). The number of microstates is defined as $W = \frac{N!}{\prod_{i} N_{i}!}$ where $N$ is the total number of particles and $N_{i}$ are the number of the -th type of particles. In our case, the system is the collection of predicted binding motifs for different TFs in a cis-regulatory sequence. We assume each TF is a different type of molecule because the DNA-binding domain of each TF belongs to a different subfamily. The number of molecular arrangements $W$ represents the number of distinguishable ways that the TFs can be ordered on the sequence. Thus, $N_{i}$ is the predicted occupancy of the $i$ -th TF and $N$ is the total predicted occupancy of all TFs on the cis-regulatory sequence. Because the predicted occupancies are continuous values, we exploit the definition of the Gamma function, $Γ (N + 1) = N!$ to rewrite $W = \frac{Γ (N + 1)}{\prod_{i} Γ (N_{i} + 1)}$ .

If we assume that each arrangement of motifs is equally likely, then we can write the probability of arrangement $w = 1, \dots, W$ as $p_{w} = \frac{1}{w}$ and rewrite the entropy as $S = - l o g (\frac{1}{w}) = - l o g (p_{w})$ , where we have dropped Boltzmann’s constant since the connection between molecular arrangements and temperature is not important. Because each arrangement is equally likely, then $\frac{1}{w}$ is also the expected value of $p_{w}$ and we can write the entropy as $S = - E [l o g (p_{w})] = - \sum w p_{w} l o g (p_{w})$ , which is Shannon entropy. By definition, Shannon entropy is also the expected value of the information content: $E [I] = - \sum w p_{w} l o g (p_{w}) = \sum w p_{w} I (w)$ where the information content $I$ of a particular state is $I (w) = l o g (p_{w})$ . Since we assumed each arrangement is equally likely, then the expected value of the information content is also the information content of each arrangement. Therefore, the information content of a cis-regulatory sequence can be written as $I = - l o g (p_{w}) = l o g W$ . We use log base 2 to express the information content in bits.

With this metric, cis-regulatory sequences with higher predicted TF occupancies generally have higher information content. Sequences with higher TF diversity have higher information content than lower diversity sequences with the same predicted occupancy. Thus, our metric captures the effects of both TF diversity and total TF occupancy. For example, consider hypothetical TFs A, B, and C. If motifs for only one TF are in a sequence, then $W$ is always one and the information content is always zero (regardless of total occupancy). The simplest case for non-zero information content is one motif for A, one motif for B, and zero motifs for C (1-1-0). Then $W = \frac{2!}{1! 1!} = 2$ and $I = 1$ bit. If we increase predicted occupancy by adding a motif for A (2-1-0), then $W = \frac{3!}{2! 1!} = 3$ and $I = 1.6$ bits, which is approximately the information content of silencers and inactive sequences. If we increase predicted occupancy again and add a second motif for B (2-2-0), then $W = \frac{4!}{2! 2!} = 6$ and $I = 2.6$ bits, which is approximately the information content of strong enhancers. If instead of increasing predicted occupancy, we instead increase diversity by replacing a motif for A with a motif for C (1-1-1), then $W = \frac{3!}{1! 1! 1!} = 6$ and once again $I = 2.6$ bits, which is higher than the lower diversity case (2-1-0).

According to 87Wunderlich and Mirny2009, the probability of observing $k$ total motifs for $m$ different TFs in a $w$ bp window is $p (k) \sim (P o i s s o n (k; λ))$ , where $λ = p m w$ and $p$ is the probability of finding a spurious motif in the genome. The expected number of windows with $k$ total motifs in a genome of length N is thus $E (k) = p (k) \cdot N$ . In mammals, $N \approx 10^{9}$ and Wunderlich and Mirny find that $p = 0.0025$ for multicellular eukaryotes. For $m = 3$ TFs and a w=164 bp window (which is the size of our sequences), $λ = 0.123$ and $E (5) = 1.6$ meaning that five total motifs for three different TFs specify an approximately unique 164 bp location in a mammalian genome. Five total motifs for three different TFs can be achieved in two ways: three motifs for A, one for B, and one for C (3-1-1), or two motifs for A, two for B, and one for C (2-2-1). In the case of 3-1-1, $W = \frac{5!}{3! 1! 1!} = 20$ and $I = 4.3$ bits. In the case of 2-2-1, $W = \frac{5!}{2! 2! 1!} = 30$ and $I = 4.9$ bits.

Machine learning

The k-mer SVM was fit using gkmSVM 19Ghandi et al.2014. All other machine learning, including cross-validation, logistic regression, and computing ROC and PR curves, was performed using scikit-learn version 0.19.1 64Pedregosa et al.2011. We wrote custom Python wrappers for gkmSVM to allow for interfacing between the C++ binaries and the rest of our workflow. We ran gkmSVM with the parameters -l 6 -k 6 -m 1. To estimate model performance, all models were fit with stratified fivefold cross-validation after shuffling the order of sequences. For the TF occupancy logistic regression model, we used L2 regularization. We selected the regularization parameter C by performing grid search with fivefold cross-validation on the values 10^–4, 10^–3, 10^–2, 10^–1, 1, 10¹, 10², 10³, 10⁴ and selecting the value that maximized the F1 score. The optimal value of C was 0.01, which we used as the regularization strength when assessing the performance of the model with other feature sets.

To assess the performance of the logistic regression model, we randomly sampled eight PWMs from the HOCOMOCO database and computed the predicted occupancy of each TF on each sequence. We then fit a new logistic regression model with these features and repeated this procedure 100 times to generate a background distribution of model performances.

To generate de novo motifs from the SVM, we generated all 6-mers and scored them against the SVM. We then ran the svmwemalign.py script from gkmSVM on the _k-mer scores with the parameters -n 10 -f 2 -m 4 and a PWM length of 6, and then used TOMTOM to compare them to the database from our motif analysis.

Other data sources

We used our previously published library 85White et al.2013 as an independent test set for our machine learning models. We defined strong enhancers as ChIP-seq peaks that were above the 95th percentile of all scrambled sequences. There was no basal promoter construct in this library, so instead we defined silencers as ChIP-seq peaks that were at least twofold below the log₂ mean of all scrambled sequences.

Previously published ChIP-seq data for NRL 23Hao et al.2012 that was re-processed by 31Hughes et al.2017 and MEF2D 2Andzelm et al.2015 was used to annotate sequences for in vivo TF binding. We converted peaks to mm10 coordinates using the UCSC liftOver tool and then used Bedtools to intersect peaks with our library.

References

1. CAlexandre
2. JPVincent
2003Requirements for transcriptional repression and activation by engrailed in Drosophila embryos130Development729739
- doi10.1242/dev.00286
- pmid12506003
1. MMAndzelm
2. TJCherry
3. DAHarmin
4. ACBoeke
5. CLee
6. MHemberg
7. BPawlyk
8. ANMalik
9. SWFlavell
10. MASandberg
11. ERaviola
12. MEGreenberg
2015MEF2D drives photoreceptor development through a genome-wide competition for tissue-specific enhancers86Neuron247263
- doi10.1016/j.neuron.2015.02.038
- pmid25801704
1. TLBailey
2. MBoden
3. FABuske
4. MFrith
5. CEGrant
6. LClementi
7. JRen
8. WWLi
9. WSNoble
2009MEME SUITE: Tools for motif discovery and searching37Nucleic Acids Research
- doi10.1093/nar/gkp335
- pmid19458158
1. SBarolo
2. JWPosakony
2002Three habits of highly effective signaling pathways: Principles of transcriptional control by developmental cell signaling16Genes & Development11671181
- doi10.1101/gad.976502
- pmid12023297
1. AHBrand
2. GMicklem
3. KNasmyth
1987A yeast silencer contains sequences that can promote autonomous plasmid replication and transcriptional activation51Cell709719
- doi10.1016/0092-8674(87)90094-8
- pmid3315230
1. SChen
2. QLWang
3. ZNie
4. HSun
5. GLennon
6. NGCopeland
7. DJGilbert
8. NAJenkins
9. DJZack
1997Crx, a novel Otx-like paired-homeodomain protein, binds to and transactivates photoreceptor cell-specific genes19Neuron10171030
- doi10.1016/s0896-6273(00)80394-3
- pmid9390516
1. JChen
2. ARattner
3. JNathans
2005The rod photoreceptor-specific nuclear receptor Nr2e3 represses transcription of multiple cone-specific genes25The Journal of Neuroscience118129
- doi10.1523/JNEUROSCI.3571-04.2005
- pmid15634773
1. CChiang
2. KAyyanathan
2013SNAIL/GFI-1 (SNAG) family zinc finger proteins in transcription regulation, chromatin dynamics, cell signaling, development, and disease24Cytokine & Growth Factor Reviews123131
- doi10.1016/j.cytogfr.2012.09.002
- pmid23102646
1. JCCorbo
2. KALawrence
3. MKarlstetter
4. CAMyers
5. MAbdelaziz
6. WDirkes
7. KWeigelt
8. MSeifert
9. VBenes
10. LGFritsche
11. BHFWeber
12. TLangmann
2010CRX ChIP-seq reveals the cis-regulatory architecture of mouse photoreceptors20Genome Research15121525
- doi10.1101/gr.109405.110
- pmid20693478
1. JCrocker
2. NAbe
3. LRinaldi
4. APMcGregor
5. NFrankel
6. SWang
7. AAlsawadi
8. PValenti
9. SPlaza
10. FPayre
11. RSMann
12. DLStern
2015Low affinity binding site clusters confer hox specificity and regulatory robustness160Cell191203
- doi10.1016/j.cell.2014.11.041
- pmid25557079
1. NDoniJayavelu
2. AJajodia
3. AMishra
4. RDHawkins
2020Candidate silencer elements for the human and mouse genomes11Nature Communications
- doi10.1038/s41467-020-14853-5
- pmid32103011
1. KMDorval
2. BPBobechko
3. HFujieda
4. SChen
5. DJZack
6. RBremner
2006Chx10 targets a subset of photoreceptor genes281The Journal of Biological Chemistry744751
- doi10.1074/jbc.M509470200
- pmid16236706
1. JErnst
2. MKellis
2012ChromHMM: automating chromatin-state discovery and characterization9Nature Methods215216
- doi10.1038/nmeth.1906
- pmid22373907
1. RFan
2. AToubal
3. SGoñi
4. KDrareni
5. ZHuang
6. FAlzaid
7. RBallaire
8. PAncel
9. NLiang
10. ADamdimopoulos
11. IHainault
12. ASoprani
13. JAron-Wisnewsky
14. FFoufelle
15. TLawrence
16. JFGautier
17. NVenteclef
18. ETreuter
2016Loss of the co-repressor GPS2 sensitizes macrophage activation upon metabolic stress induced by obesity and type 2 diabetes22Nature Medicine780791
- doi10.1038/nm.4114
- pmid27270589
1. EKFarley
2. KMOlson
3. WZhang
4. AJBrandt
5. DSRokhsar
6. MSLevine
2015Suboptimization of developmental enhancers350Science325328
- doi10.1126/science.aac6948
- pmid26472909
1. EKFarley
2. KMOlson
3. WZhang
4. DSRokhsar
5. MSLevine
2016Syntax compensates for poor binding sites to encode tissue specificity of developmental enhancers113PNAS65086513
- doi10.1073/pnas.1605085113
- pmid27155014
1. CLFreund
2. CYGregory-Evans
3. TFurukawa
4. MPapaioannou
5. JLooser
6. LPloder
7. JBellingham
8. DNg
9. JASHerbrick
10. ADuncan
11. SWScherer
12. LCTsui
13. ALoutradis-Anagnostou
14. SGJacobson
15. CLCepko
16. SSBhattacharya
17. RRMcInnes
1997Cone-rod dystrophy due to mutations in a novel photoreceptor-specific homeobox gene (CRX) essential for maintenance of the photoreceptor91Cell543553
- doi10.1016/s0092-8674(00)80440-7
- pmid9390563
1. TFurukawa
2. EMMorrow
3. CLCepko
1997Crx, a novel otx-like homeobox gene, shows photoreceptor-specific expression and regulates photoreceptor differentiation91Cell531541
- doi10.1016/s0092-8674(00)80439-0
- pmid9390562
1. MGhandi
2. DLee
3. MMohammad-Noori
4. MABeer
2014Enhanced Regulatory Sequence Prediction Using Gapped k-mer Features10PLOS Computational Biology
- doi10.1371/journal.pcbi.1003711
- pmid25033408
1. SSGisselbrecht
2. APalagi
3. JKurland
4. JMRogers
5. HOzadam
6. YZhan
7. JDekker
8. MLBulyk
2020Transcriptional silencers in Drosophila serve a dual role as transcriptional enhancers in alternate cellular contexts77Molecular Cell324337
- doi10.1016/j.molcel.2019.10.004
- pmid31704182
1. JAGrass
2. MEBoyer
3. SPal
4. JWu
5. MJWeiss
6. EHBresnick
2003GATA-1-dependent transcriptional repression of GATA-2 via disruption of positive autoregulation and domain-wide chromatin remodeling100PNAS88118816
- doi10.1073/pnas.1432147100
- pmid12857954
1. MHaeussler
2. ASZweig
3. CTyner
4. MLSpeir
5. KRRosenbloom
6. BJRaney
7. CMLee
8. BTLee
9. ASHinrichs
10. JNGonzalez
11. DGibson
12. MDiekhans
13. HClawson
14. JCasper
15. GPBarber
16. DHaussler
17. RMKuhn
18. WJKent
2019The UCSC Genome Browser database: 2019 update47Nucleic Acids Research
- doi10.1093/nar/gky1095
- pmid30407534
1. HHao
2. DSKim
3. BKlocke
4. KRJohnson
5. KCui
6. NGotoh
7. CZang
8. JGregorski
9. LGieser
10. WPeng
11. YFann
12. MSeifert
13. KZhao
14. ASwaroop
2012Transcriptional regulation of rod photoreceptor homeostasis revealed by in vivo NRL targetome analysis8PLOS Genetics
- doi10.1371/journal.pgen.1002649
- pmid22511886
1. CRHarris
2. KJMillman
3. SJvanderWalt
4. RGommers
5. PVirtanen
6. DCournapeau
7. EWieser
8. JTaylor
9. SBerg
10. NJSmith
11. RKern
12. MPicus
13. SHoyer
14. MHvanKerkwijk
15. MBrett
16. AHaldane
17. JFDelRío
18. MWiebe
19. PPeterson
20. PGérard-Marchant
21. KSheppard
22. TReddy
23. WWeckesser
24. HAbbasi
25. CGohlke
26. TEOliphant
2020Array programming with NumPy585Nature357362
- doi10.1038/s41586-020-2649-2
- pmid32939066
1. AKHennig
2. GHPeng
3. SChen
2008Regulation of photoreceptor gene expression by Crx-associated transcription factor network1192Brain Research114133
- doi10.1016/j.brainres.2007.06.036
- pmid17662965
1. JHlawatsch
2. MKarlstetter
3. AAslanidis
4. ALückoff
5. YWalczak
6. MPlank
7. JBöck
8. TLangmann
2013Sterile alpha motif containing 7 (SAMD7) is a novel Crx-regulated transcriptional repressor in the retina8PLOS ONE
- doi10.1371/journal.pone.0060633
- pmid23565263
1. MMHoffman
2. OJBuske
3. JWang
4. ZWeng
5. JABilmes
6. WSNoble
2012Unsupervised pattern discovery in human chromatin structure through genomic segmentation9Nature Methods473476
- doi10.1038/nmeth.1937
- pmid22426492
1. THCHsiau
2. CDiaconu
3. CAMyers
4. JLee
5. CLCepko
6. JCCorbo
2007The cis-regulatory logic of the mammalian photoreceptor transcriptional network2PLOS ONE
- doi10.1371/journal.pone.0000643
- pmid17653270
1. DHuang
2. HMPetrykowska
3. BFMiller
4. LElnitski
5. IOvcharenko
2019Identification of human silencers by correlating cross-tissue epigenetic profiles and gene expression29Genome Research657667
- doi10.1101/gr.247007.118
- pmid30886051
1. ZHuang
2. NLiang
3. SGoñi
4. ADamdimopoulos
5. CWang
6. RBallaire
7. JJager
8. HNiskanen
9. HHan
10. TJakobsson
11. APBracken
12. MAouadi
13. NVenteclef
14. MUKaikkonen
15. RFan
16. ETreuter
2021The corepressors GPS2 and SMRT control enhancer and silencer remodeling via eRNA transcription during inflammatory activation of macrophages81Molecular Cell953968
- doi10.1016/j.molcel.2020.12.040
- pmid33503407
1. AEOHughes
2. JMEnright
3. CAMyers
4. SQShen
5. JCCorbo
2017Cell type-specific epigenomic analysis reveals a uniquely closed chromatin architecture in mouse rod photoreceptors7Scientific Reports
- doi10.1038/srep43184
- pmid28256534
1. AEOHughes
2. CAMyers
3. JCCorbo
2018A massively parallel reporter assay reveals context-dependent activity of homeodomain binding sites in vivo28Genome Research15201531
- doi10.1101/gr.231886.117
- pmid30158147
1. JDHunter
2007Matplotlib: A 2D Graphics Environment9Computing in Science & Engineering9095
- doi10.1109/MCSE.2007.55
1. SIrie
2. RSanuki
3. YMuranishi
4. KKato
5. TChaya
6. TFurukawa
2015Rax Homeoprotein Regulates Photoreceptor Cell Maturation and Survival in Association with Crx in the Postnatal Mouse Retina35Molecular and Cellular Biology25832596
- doi10.1128/MCB.00048-15
- pmid25986607
1. TIype
2. DGTaylor
3. SMZiesmann
4. JCGarmey
5. HWatada
6. RGMirmira
2004The transcriptional repressor Nkx6.1 also functions as a deoxyribonucleic acid context-dependent transcriptional activator during pancreatic beta-cell differentiation: evidence for feedback activation of the nkx6.1 gene by Nkx6.118Molecular Endocrinology13631375
- doi10.1210/me.2004-0006
- pmid15056733
1. LJia
2. ECTOh
3. LNg
4. MSrinivas
5. MBrooks
6. ASwaroop
7. DForrest
2009Retinoid-related orphan nuclear receptor RORbeta is an early-acting factor in rod photoreceptor development106PNAS1753417539
- doi10.1073/pnas.0902425106
- pmid19805139
1. JJiang
2. HCai
3. QZhou
4. MLevine
1993Conversion of a dorsal-dependent silencer into an enhancer: evidence for dorsal corepressors12The EMBO Journal32013209
- doi10.1002/j.1460-2075.1993.tb05989.x
- pmid8344257
1. KDJohnson
2. SIKim
3. EHBresnick
2006Differential sensitivities of transcription factor target genes underlie cell type-specific gene expression profiles103PNAS1593915944
- doi10.1073/pnas.0604041103
- pmid17043224
1. GJunion
2. MSpivakov
3. CGirardot
4. MBraun
5. EHGustafson
6. EBirney
7. EEMFurlong
2012A transcription factor collective defines cardiac cell fate and reflects lineage history148Cell473486
- doi10.1016/j.cell.2012.01.030
- pmid22304916
1. BKJustin
2021Logomaker
1. DRKelley
2. JSnoek
3. JLRinn
2016Basset: Learning the regulatory code of the accessible genome with deep convolutional neural networks26Genome Research990999
- doi10.1101/gr.200535.115
- pmid27197224
1. AKhan
2. OFornes
3. AStigliani
4. MGheorghe
5. JACastro-Mondragon
6. RvanderLee
7. ABessy
8. JChèneby
9. SRKulkarni
10. GTan
11. DBaranasic
12. DJArenillas
13. ASandelin
14. KVandepoele
15. BLenhard
16. BBallester
17. WWWasserman
18. FParcy
19. AMathelier
2018JASPAR 2018: update of the open-access database of transcription factor binding profiles and its web framework46Nucleic Acids Research
- doi10.1093/nar/gkx1188
- pmid29161433
1. AKimura
2. DSingh
3. EFWawrousek
4. MKikuchi
5. MNakamura
6. TShinohara
2000Both PCE-1/RX and OTX/CRX interactions are necessary for photoreceptor-specific gene expression275The Journal of Biological Chemistry11521160
- doi10.1074/jbc.275.2.1152
- pmid10625658
1. SLKlemm
2. ZShipony
3. WJGreenleaf
2019Chromatin accessibility and the regulatory epigenome20Nature Reviews. Genetics207220
- doi10.1038/s41576-018-0089-8
- pmid30675018
1. CKoike
2. ANishida
3. SUeno
4. HSaito
5. RSanuki
6. SSato
7. AFurukawa
8. SAizawa
9. IMatsuo
10. NSuzuki
11. MKondo
12. TFurukawa
2007Functional roles of Otx2 transcription factor in postnatal mouse retinal development27Molecular and Cellular Biology83188329
- doi10.1128/MCB.01209-07
- pmid17908793
1. IVKulakovskiy
2. IEVorontsov
3. ISYevshin
4. RNSharipov
5. ADFedorova
6. EIRumynskiy
7. YAMedvedeva
8. AMagana-Mora
9. VBBajic
10. DAPapatsenko
11. FAKolpakov
12. VJMakeev
2018HOCOMOCO: Towards a complete collection of transcription factor binding models for human and mouse via large-scale ChIP-seq analysis46Nucleic Acids Research
- doi10.1093/nar/gkx1106
- pmid29140464
1. JCKwasnieski
2. IMogno
3. CAMyers
4. JCCorbo
5. BACohen
2012Complex effects of nucleotide variants in a mammalian cis-regulatory element109PNAS1949819503
- doi10.1073/pnas.1210678109
- pmid23129659
1. JCKwasnieski
2. CFiore
3. HGChaudhari
4. BACohen
2014High-throughput functional testing of ENCODE segmentation predictions24Genome Research15951602
- doi10.1101/gr.173518.114
- pmid25035418
1. JLee
2. CAMyers
3. NWilliams
4. MAbdelaziz
5. JCCorbo
2010Quantitative fine-tuning of photoreceptor cis-regulatory elements through affinity modulation of transcription factor binding sites17Gene Therapy13901399
- doi10.1038/gt.2010.77
- pmid20463752
1. DLee
2. RKarchin
3. MABeer
2011Discriminative prediction of mammalian enhancers from DNA sequence21Genome Research21672180
- doi10.1101/gr.121905.111
- pmid21875935
1. LELerner
2. GHPeng
3. YEGribanova
4. SChen
5. DBFarber
2005Sp4 is expressed in retinal neurons, activates transcription of photoreceptor-specific genes, and synergizes with Crx280The Journal of Biological Chemistry2064220650
- doi10.1074/jbc.M500957200
- pmid15781457
1. YRLiu
2. ZALaghari
3. CANovoa
4. JHughes
5. JRMWebster
6. PEGoodwin
7. SPWheatley
8. PJScotting
2014Sox2 acts as a transcriptional repressor in neural stem cells15BMC Neuroscience
- doi10.1186/1471-2202-15-95
- pmid25103589
1. FMartínez-Montañés
2. ARienzo
3. DPoveda-Huertes
4. APascual-Ahuir
5. MProft
2013Activator and repressor functions of the Mot3 transcription factor in the osmostress response of Saccharomyces cerevisiae12Eukaryotic Cell636647
- doi10.1128/EC.00037-13
- pmid23435728
1. WMcKinney
2010Proceedings of the 9th Python in Science conference5156
1. AJMears
2. MKondo
3. PKSwain
4. YTakada
5. RABush
6. TLSaunders
7. PASieving
8. ASwaroop
2001Nrl is required for rod photoreceptor development29Nature Genetics447452
- doi10.1038/ng774
- pmid11694879
1. KPMitton
2. PKSwain
3. SChen
4. SXu
5. DJZack
6. ASwaroop
2000The leucine zipper of NRL interacts with the CRX homeodomain. A possible mechanism of transcriptional synergy in rhodopsin regulation275The Journal of Biological Chemistry2979429799
- doi10.1074/jbc.M003658200
- pmid10887186
1. KPMitton
2. PKSwain
3. HKhanna
4. MDowd
5. IJApel
6. ASwaroop
2003Interaction of retinal bZIP transcription factor NRL with Flt3-interacting zinc-finger protein Fiz1: possible role of Fiz1 as a transcriptional repressor12Human Molecular Genetics365373
- doi10.1093/hmg/ddg035
- pmid12566383
1. JEMoore
2. MJPurcaro
3. HEPratt
4. CBEpstein
5. NShoresh
6. JAdrian
7. TKawli
8. CADavis
9. ADobin
10. RKaul
11. JHalow
12. ELVanNostrand
13. PFreese
14. DUGorkin
15. YShen
16. YHe
17. MMackiewicz
18. FPauli-Behn
19. BAWilliams
20. AMortazavi
21. CAKeller
22. XOZhang
23. SIElhajjajy
24. JHuey
25. DEDickel
26. VSnetkova
27. XWei
28. XWang
29. JCRivera-Mulia
30. JRozowsky
31. JZhang
32. SBChhetri
33. JZhang
34. AVictorsen
35. KPWhite
36. AVisel
37. GWYeo
38. CBBurge
39. ELécuyer
40. DMGilbert
41. JDekker
42. JRinn
43. EMMendenhall
44. JREcker
45. MKellis
46. RJKlein
47. WSNoble
48. AKundaje
49. RGuigó
50. PJFarnham
51. JMCherry
52. RMMyers
53. BRen
54. BRGraveley
55. MBGerstein
56. LAPennacchio
57. MPSnyder
58. BEBernstein
59. BWold
60. RCHardison
61. TRGingeras
62. JAStamatoyannopoulos
63. ZWeng
64. ENCODE Project Consortium
2020Expanded encyclopaedias of DNA elements in the human and mouse genomes583Nature699710
- doi10.1038/s41586-020-2493-4
- pmid32728249
1. EMMorrow
2. TFurukawa
3. JELee
4. CLCepko
1999NeuroD regulates multiple functions in the developing neural retina in rodent126Development2336
- doi10.1242/dev.126.1.23
- pmid9834183
1. DPMurphy
2. AEHughes
3. KALawrence
4. CAMyers
5. JCCorbo
2019Cis-regulatory basis of sister cell type divergence in the vertebrate retina8eLife
- doi10.7554/eLife.48216
- pmid31633482
1. CYNgan
2. CHWong
3. HTjong
4. WWang
5. RLGoldfeder
6. CChoi
7. HHe
8. LGong
9. JLin
10. BUrban
11. JChow
12. MLi
13. JLim
14. VPhilip
15. SAMurray
16. HWang
17. CLWei
2020Chromatin interaction analyses elucidate the roles of PRC2-bound silencers in mouse development52Nature Genetics264272
- doi10.1038/s41588-020-0581-x
- pmid32094912
1. BPang
2. MPSnyder
2020Systematic identification of silencers in human cells52Nature Genetics254263
- doi10.1038/s41588-020-0578-5
- pmid32094911
1. DSParker
2. MAWhite
3. ARamos
4. BACohen
5. SBarolo
2011The cis-regulatory logic of Hedgehog gradient responses: key roles for gli binding affinity, competition, and cooperativity4Science Signaling
- doi10.1126/scisignal.2002077
1. FPedregosa
2. GVaroquaux
3. AGramfort
4. VMichel
5. BThirion
6. OGrisel
7. MBlondel
8. PPrettenhofer
9. RWeiss
10. VDubourg
2011Scikit-learn: Machine learning in Python12The Journal of Machine Learning Research28252830
1. GHPeng
2. OAhmad
3. FAhmad
4. JLiu
5. SChen
2005The photoreceptor-specific nuclear receptor Nr2e3 interacts with CRX and exerts opposing effects on the transcription of rod versus cone genes14Human Molecular Genetics747764
- doi10.1093/hmg/ddi070
- pmid15689355
1. HMPetrykowska
2. CMVockley
3. LElnitski
2008Detection and characterization of silencers and enhancer-blockers in the greater CFTR locus18Genome Research12381246
- doi10.1101/gr.073817.107
- pmid18436892
1. RPhillips
2. JKondev
3. JTheriot
4. HGGarcia
5. NOrme
2012Physical Biology of the CellGarland Science
- doi10.1201/9781134111589
1. ARQuinlan
2. IMHall
2010Bedtools: A flexible suite of utilities for comparing genomic features26Bioinformatics841842
- doi10.1093/bioinformatics/btq033
- pmid20110278
1. IRachmin
2. EAmsalem
3. EGolomb
4. RBeeri
5. DGilon
6. PFang
7. HNechushtan
8. GKay
9. MGuo
10. PLYiqing
11. RSYFoo
12. DEFisher
13. ERazin
14. STshori
2015FHL2 switches MITF from activator to repressor of Erbin expression during cardiac hypertrophy195International Journal of Cardiology8594
- doi10.1016/j.ijcard.2015.05.108
- pmid26025865
1. JRister
2. ARazzaq
3. PBoodram
4. NDesai
5. CTsanis
6. HChen
7. DJukam
8. CDesplan
2015Single-base pair differences in a shared motif determine differential Rhodopsin expression350Science12581261
- doi10.1126/science.aab3417
- pmid26785491
1. JERoger
2. AHiriyanna
3. NGotoh
4. HHao
5. DFCheng
6. RRatnapriya
7. MAIKautzmann
8. BChang
9. ASwaroop
2014OTX2 loss causes rod differentiation defect in CRX-associated congenital blindness124The Journal of Clinical Investigation631643
- doi10.1172/JCI72722
- pmid24382353
1. PARuzycki
2. XZhang
3. SChen
2018CRX directs photoreceptor differentiation by accelerating chromatin remodeling at specific target sites11Epigenetics & Chromatin
- doi10.1186/s13072-018-0212-2
- pmid30068366
1. MAHSamee
2. BGBruneau
3. KSPollard
2019A De Novo Shape Motif Discovery Algorithm Reveals Preferences of Transcription Factors for DNA Shape Beyond Sequence Motifs8Cell Systems2742
- doi10.1016/j.cels.2018.12.001
- pmid30660610
1. MAHSamee
2021Shape-motif
1. RSanuki
2. YOmori
3. CKoike
4. SSato
5. TFurukawa
2010Panky, a novel photoreceptor-specific ankyrin repeat protein, is a transcriptional cofactor that suppresses CRX-regulated photoreceptor genes584FEBS Letters753758
- doi10.1016/j.febslet.2009.12.030
- pmid20026326
1. JASegert
2. SSGisselbrecht
3. MLBulyk
2021Transcriptional silencers: Driving gene expression with the brakes on37Trends in Genetics514527
- doi10.1016/j.tig.2021.02.002
- pmid33712326
1. ASethi
2. MGu
3. EGumusgoz
4. LChan
5. K-KYan
6. JRozowsky
7. IBarozzi
8. VAfzal
9. JAAkiyama
10. IPlajzer-Frick
11. CYan
12. CSNovak
13. MKato
14. THGarvin
15. QPham
16. AHarrington
17. BJMannion
18. EALee
19. YFukuda-Yuzawa
20. AVisel
21. DEDickel
22. KYYip
23. RSutton
24. LAPennacchio
25. MGerstein
2020Supervised enhancer prediction with epigenetic pattern recognition and targeted validation17Nature Methods807814
- doi10.1038/s41592-020-0907-8
- pmid32737473
1. FSpitz
2. EEMFurlong
2012Transcription factors: From enhancer binding to developmental control13Nature Reviews. Genetics613626
- doi10.1038/nrg3207
- pmid22868264
1. MSrinivas
2. LNg
3. HLiu
4. LJia
5. DForrest
2006Activation of the blue opsin gene in cone photoreceptor development by retinoid-related orphan receptor beta20Molecular Endocrinology17281741
- doi10.1210/me.2005-0505
- pmid16574740
1. GStampfel
2. TKazmar
3. OFrank
4. SWienerroither
5. FReiter
6. AStark
2015Transcriptional regulators form diverse groups with context-dependent regulatory functions528Nature147151
- doi10.1038/nature15545
- pmid26550828
1. ATareen
2. JBKinney
2020Logomaker: beautiful sequence logos in Python36Bioinformatics22722274
- doi10.1093/bioinformatics/btz921
- pmid31821414
1. PVirtanen
2. RGommers
3. TEOliphant
4. MHaberland
5. TReddy
6. DCournapeau
7. EBurovski
8. PPeterson
9. WWeckesser
10. JBright
11. SJvanderWalt
12. MBrett
13. JWilson
14. KJMillman
15. NMayorov
16. ARJNelson
17. EJones
18. RKern
19. ELarson
20. CJCarey
21. İPolat
22. YFeng
23. EWMoore
24. JVanderPlas
25. DLaxalde
26. JPerktold
27. RCimrman
28. IHenriksen
29. EAQuintero
30. CRHarris
31. AMArchibald
32. AHRibeiro
33. FPedregosa
34. PvanMulbregt
35. SciPy 1.0 Contributors
2020SciPy 1.0: Fundamental algorithms for scientific computing in Python17Nature Methods261272
- doi10.1038/s41592-019-0686-2
- pmid32015543
1. SWang
2. CSengel
3. MMEmerson
4. CLCepko
2014A gene regulatory network controls the binary fate decision of rod and bipolar cells in the vertebrate retina30Developmental Cell513527
- doi10.1016/j.devcel.2014.07.018
- pmid25155555
1. ALWebber
2. PHodor
3. CJThut
4. TFVogt
5. TZhang
6. DJHolder
7. KPetrukhin
2008Dual role of Nr2e3 in photoreceptor development and maintenance87Experimental Eye Research3548
- doi10.1016/j.exer.2008.04.006
- pmid18547563
1. MAWhite
2. CAMyers
3. JCCorbo
4. BACohen
2013Massively parallel in vivo enhancer assay reveals that highly local features determine the cis-regulatory function of ChIP-seq peaks110PNAS1195211957
- doi10.1073/pnas.1307449110
- pmid23818646
1. MAWhite
2. JCKwasnieski
3. CAMyers
4. SQShen
5. JCCorbo
6. BACohen
2016A Simple Grammar Defines Activating and Repressing cis-Regulatory Elements in Photoreceptors17Cell Reports12471254
- doi10.1016/j.celrep.2016.09.066
- pmid27783940
1. ZWunderlich
2. LAMirny
2009Different gene regulation strategies revealed by analysis of binding motifs25Trends in Genetics434440
- doi10.1016/j.tig.2009.08.003
- pmid19815308
1. ZYang
2. KDing
3. LPan
4. MDeng
5. LGan
2003Math5 determines the competence state of retinal ganglion cell progenitors264Developmental Biology240254
- doi10.1016/j.ydbio.2003.08.005
- pmid14623245
1. YZhao
2. DGranas
3. GDStormo
2009Inferring binding energies from selected binding sites5PLOS Computational Biology
- doi10.1371/journal.pcbi.1000590
- pmid19997485
1. JZhou
2. OGTroyanskaya
2015Predicting effects of noncoding variants with deep learning-based sequence model12Nature Methods931934
- doi10.1038/nmeth.3547
- pmid26301843