Multimodal protein representation learning and target-aware variational auto-encoders for protein-binding ligand generation

Without knowledge of specific pockets, generating ligands based on the global structure of a protein target plays a crucial role in drug discovery as it helps reduce the search space for potential drug-like candidates in the pipeline. However, contemporary methods require optimizing tailored networks for each protein, which is arduous and costly. To address this issue, we introduce TargetVAE, a target-aware variational auto-encoder that generates ligands with desirable properties including high binding affinity and high synthesizability to arbitrary target proteins, guided by a multimodal deep neural network built based on geometric and sequence models, named Protein Multimodal Network (PMN), as the prior for the generative model. PMN unifies different representations of proteins (e.g. primary structure—sequence of amino acids, 3D tertiary structure, and residue-level graph) into a single representation. Our multimodal architecture learns from the entire protein structure and is able to capture their sequential, topological, and geometrical information by utilizing language modeling, graph neural networks, and geometric deep learning. We showcase the superiority of our approach by conducting extensive experiments and evaluations, including predicting protein-ligand binding affinity in the PBDBind v2020 dataset as well as the assessment of generative model quality, ligand generation for unseen targets, and docking score computation. Empirical results demonstrate the promising and competitive performance of our proposed approach. Our software package is publicly available at https://github.com/HySonLab/Ligand_Generation.


Introduction
Drug discovery is a complex and expensive process that involves multiple stages and often takes years of development, with costs running into billions of dollars [1]. The first stage is to design novel drug-like compounds with high binding affinities to target proteins.This process consists of two sub-tasks: searching for candidates and measuring drug-target affinities (DTA).Searching for potential candidates in a huge database of roughly 10 33 chemically valid molecules is a daunting task as current methods often rely on virtual screenings, professional software, and expert evaluation [2,3].Besides, DTA are critical measurements for identifying potential candidates and avoiding those that are inefficient for clinical trials.The most reliable technique for predicting DTA involves atomistic molecular dynamics simulations.However, these methods are computationally expensive and time-consuming, making them infeasible for large-scale sets of protein-ligand complexes.Our ultimate objective is to accelerate and automate these two sub-tasks in the first stage of the drug development process, using computational methods and machine-learning techniques.
DGMs have demonstrated remarkable results in unconditionally generating or optimizing molecular properties, such as drug-likeness (QED) and synthetic accessibility (SA).However, they are prohibitively slow when enhancing binding affinity or other computationally expensive molecular properties.This is because they need to be trained in reinforcement learning frameworks where the generated molecular graph is modified based on a reward function determined by calling a property network that estimates the binding affinities.While effective and powerful, these approaches require specific property networks to be trained for each target protein, which is challenging due to the vast number of known and unknown proteins [3,12].Moreover, binding scores (i.e.labels for supervised learning) are not widely available and time-consuming to approximate with software such as Autodock Vina [13].Additionally, several existing studies [14][15][16] incorporate prior knowledge of protein structures to guide conditional generative models to generate bioactive molecules.However, these methods rely solely on the information of specific binding sites of target proteins and are limited when the binding sites are not determined.It is worth noting that determining binding sites, also known as pockets, on a target protein (receptor) is a challenging problem, which consists of many constraints [17].
Proteins are macromolecules that can be represented in terms of sequences of amino acids (i.e.primary structure), 2D graphs at residue level constructed by nearest neighbors from folding information (i.e.tertiary structure), or 3D point clouds at atom level.Recent advanced methods for protein representation learning leverage language models, graph neural networks (GNNs), and convolutional neural networks (CNNs).In sequence-based methods [18][19][20][21][22], a protein sequence is regarded as a long sequence of tokens (i.e.k-mers [18]) that are fed to a transformer-based language model.In contrast, GNNs and CNNs-based approaches [23][24][25][26][27] operate on relational and geometric structures of proteins, respectively.While sequence-based approaches can capture the relationships among distant residues in a long protein sequence, they are not able to exploit the geometric relations among them.On the other hand, although GNNs and CNNs can learn spatial information about protein structures, they are limited in their ability to capture long-range interactions in large protein structures due to their reliance on localities.Recent years have also seen the rise of pre-trained large language models for scientific discovery, which have achieved remarkable results in protein science [28,29].This motivates future work to apply language models to many downstream tasks in drug discovery, where receptor representations are essential for the search for novel ligands.

Contributions
This work aims at developing a data-driven approach that can help accelerate the drug discovery process.We develop a conditional DGM to generate drug-like ligands that can bind to a target protein when its binding sites are unknown.This approach facilitates the process of determining binding sites in drug discovery, which is computationally expensive.Furthermore, we build a multimodal protein network to leverage multiple data types of proteins.In summary, our contributions are two-fold as follows: • We build a conditional VAE model, named TargetVAE, that can generate chemically valid, drug-like molecules with high binding scores to an arbitrarily given protein structure.Apart from other methods, ours can directly condition on the entire structure of any protein target and design multiple candidates that can bind to it, without requiring the training of a specific property network for each target.It is important to note that our generative model is conditional on the whole protein structure rather than a specific pocket or binding site (i.e. a region of the protein surface where the ligand binds to) as in works of Luo et al [16], Guan et al [30], Peng et al [31], Liu et al [32].Our approach is more flexible because a protein complex can contain multiple binding sites and therefore potentially have different ligands.TargetVAE allows us to efficiently generate ligand candidates with high binding affinity without prior knowledge of any binding site.• We design a novel architecture, named Protein Multimodal Network (PMN), that unifies different modalities of proteins, i.e. sequence of amino-acids and 3D tertiary structure, to improve the performance of predicting binding affinities.The proposed model shows competitive results on the PDBBind v2020 benchmark in comparison with current state-of-the-art methods.Our novel multimodal architecture enables us to efficiently produce a protein embedding that can serve as the prior for our generative model (i.e.TargetVAE), and accurately estimate protein-ligand binding affinity, and potentially replace the computationally expensive molecular dynamical simulation in the evaluation of ligand generation.

Related work 2.1. Protein-ligand binding prediction
Machine learning methods, especially GNNs, have emerged as effective techniques for binding affinity prediction [33].Using a Kronecker regularized least squares approach (KronRLS), Nascimento et al [34] cast the problem as a link prediction task on bipartite networks and compute distinct kernels that indicate the similarities among drugs and targets to make predictions.Apart from binary prediction, He et al [35] introduce a framework named SimBoost that can predict continuous binding affinity scores between drugs and targets.In the deep learning era, Öztürk et al [36] propose DeepDTA, a deep-learning-based method that uses CNNs to operate on sequence representations of drugs and protein targets.Moreover, Zhao et al [37] use generative adversarial networks to learn better feature representations for compounds and proteins.
On the other hand, Nguyen et al [38]; Voitsitskyi et al [39] utilize GNNs to learn the representations of the molecular graphs and protein structures, which are superior to the previous methods operating on sequences and handcrafted features.However, the common limitation of these above approaches is that none of them covers a wide range of representations of proteins.Indeed, our proposed method is the first multimodal model combining sequence, graph, and spatial information of proteins altogether.

Molecule generation
Previous studies on molecule generation are mostly categorized into SMILES-based and graph-based approaches [40].Gómez-Bombarelli et al [41][42][43] use recurrent neural networks to build generative models operating on SMILES strings.However, these SMILES-based approaches often generate chemically invalid molecules.Kusner et al [44] and Dai et al [45] circumvent this issue by augmenting the decoders with grammar and semantic constraints to only generate valid molecules, yet this added information does not fully capture chemical validity in the latent space.Apart from SMILES-based methods, several works [5-10, 46, 47] propose graph generative models to design novel drug-like molecules.For example, Jin et al [5]; Jin et al [6] can generate molecules with 100% validity, but their methods are relatively slow as chemical rules are verified during the generative process.Although graph-based methods perform well in unconditional molecule generation, they have difficulty generating molecules with optimized properties.For a target-aware generation, reinforcement learning methods are used to systematically modify the generated molecular graphs [4][5][6][7].Distinguishing from other prior works based on SMILES representation, we utilize the recently proposed SELFIES representation [48] to achieve a high chemical validity in generated ligands.Given the success of conditional VAE in image generation [49], our work is the first attempt to introduce a learnable prior based on the whole protein structures.

PMN
Proteins are complex structures that consist of long chains of residues/amino acids.Each amino acid is a molecule with 3D structures, and a combination of hundreds to thousands of residues determines the unique 3D structure of a specific protein and its functions.It is worth noting that while two residues are distant along the protein sequence, they could be close to each other in three-dimensional space.This is our key observation to design a novel framework that can unify different representations of proteins in an end-to-end learning manner.
In the field of graph learning, the conventional GNNs based on the message passing scheme [50] that propagates and aggregates information of each node to and from its local neighborhoods are incapable of capturing the long-range interactions in a large-diameter or long-range4 graph [51,52] while suffering from over-smoothing [53] and over-squashing [54] problems.Furthermore, to obtain a global understanding of the whole input graph, the message-passing scheme needs the number of layers/iterations proportional to the diameter length for distant nodes to 'communicate' with each other.That is computationally infeasible for macromolecules with thousands of atoms or residues like proteins.Meanwhile, the graph Transformers that consider all pairwise node interactions via the self-attention mechanism can successfully capture the long-range dependencies [52,55,56].Since proteins can be seen as long-range graphs, we utilize sequential and graph Transformers to encode both sequences and 3D graphs of residues and combine them to create a unified representation for a large protein, making our model operate on multi-modalities of proteins.Our Transformer-based model can efficiently capture a protein's local and global information with a reasonably small number of layers.

Geometric learning on protein
According to figure 1, there are three major components in the 3D modeling part, including a local encoder, a geometric vector perceptron (GVP) module [26], and a global Transformers encoder.

Local message-passing
We use a message-passing network (MPNN) in which the GVPs [26] replace dense layers to operate on invariant features: where ⊕ is the concatenation; m ij computed by a module of three GVP layers denotes the message propagated from node j to i. Also, h v and h indicate the embeddings of node i and edge (j → i) and are tuples of scalar and vector features as described appendix A.1.The local encoder outputs a tuple of scalar and vector features for each residue node, which are rotationally invariant and equivariant, respectively.

Global interaction
We utilize a GVP module to update the tuple , and we take the invariant scalar feature s ′ ∈ R d as the node embedding for successor modules.The resulting tensor S ∈ R N×d , in which row i indicates a d-dimensional scalar feature s i of node i, is passed to a L-layer Transformers encoder: Here, we initialize Z 0 ≜ S; and we have as learnable weight matrices/parameters corresponding to the query Q ℓ , key K ℓ and value V ℓ matrices, respectively, of the Transformer at each layer ℓ; and Z g ≜ Z L denotes the final node embeddings produced by the network.Notably, this global encoder allows residue nodes to attend to other nodes on a large protein graph, especially those that are distant from them (i.e.long-range modeling).Finally, we aggregate node embeddings by a row-wise Aggregator ζ (e.g.mean, max, sum, etc) to produce an embedding for the protein structure

Language modeling on protein sequence
A protein can be represented as a sequence s = (s 0 , s 1 , . . ., s n ) in which s i ∈ R 20 is a one-hot vector indicating one in a total of 20 types of residues.We utilize Transformer-based language models, where the layers are the same as in equations ( 3)-( 5), to compute the text representation of this protein sequence with the initial embeddings Here, p i is the positional encoding feature added at each token i.In this work, for the language modeling component, we use the pre-trained Transformer protein language models proposed by Lin et al [29] to extract protein-level embeddings for each protein.Then, we define p s = ζ(Z s ) ∈ R d as the global representation for the entire protein sequence.

Multi-modal fusion
Finally, to produce the final representations of proteins, we concatenate their geometric and sequential features from the Transformers-based models and process them by a feed-forward network: Here, ⊕ denotes the concatenation between two global vector features p s and p g .

Problem setup
Given a dataset D of protein-ligand pairs, our objective is to predict the binding affinity and generate novel drug-like ligands that have the potential to bind to a conditioning protein structure.We cast the former as a prediction task based on geometric and relational reasoning on protein and ligand structures, whereas the latter is regarded as a protein-structure conditioned ligand generation.Let (l, p, s) ∈ D be a pair of protein-ligand where l and p denote the representations of ligands and proteins, respectively, and s indicates the binding score between them.Additionally, figures 2 and 3 depict the overview of our approach in both tasks.

Binding affinity prediction
Figure 2 illustrates our designed architecture for predicting the binding affinities between ligands and their target proteins.Regarding the ligand part, we represent small molecules as 2D graphs G = (V, E) where nodes v ∈ V are the atoms and edges e ∈ E indicate their covalence bonds.Then, we use graph attention networks (GAT), a MPNN, to learn their representations.In particular, at the layer t, the embedding vector x t i of node i ∈ V is updated as: where the attention coefficients α i,j are computed as Here, σ denotes non-linear activations, N (i) is the set of neighbors of node i, and ⊕ denotes concatenation, and a and W are the weight vector and matrix, respectively.At the end of the network, we aggregate all the node embeddings on the molecular graphs to produce its global embedding as ℓ = ζ{x i |x i ∈ V}.In addition to graph-based features, Morgan fingerprints can also be used as molecule features.Finally, we combine features from all sources, including sequence, geometric information of proteins, and graph-based features of ligands, and pass them to a feed-forward network (FFN) to make predictions:

ŷ = ϕ (h) . (10)
Here, h is the combination of all sources of inputs, and ϕ is a feed-forward network that predicts the binding affinity score ŷ.

Target-aware ligand generation
Although there exist many machine-learning approaches that generate drug-like molecules, it is challenging for graph-based or smiles-based methods to generate chemically valid ligands with high probability.Meanwhile, SELFIES (SELF-referenced Embedded Strings) [48] is a string-based representation of molecules 100% robust to molecular validity.A ligand l can be defined as a string of l = (l 1 , l 2 , . . ., l n ) in which l i is a SELFIES token, which belongs to a predefined symbol set S derived from the training dataset.We generate ligands l = ( l1 , l2 , . . ., ln ) by computing n independent probability vectors y = (y 1 , y 2 , . . ., y n ), y i ∈ R |S| .Each new token li is defined as li = S j where j = argmax 0⩽j <|S| (y i ).
Let ϕ, θ, and ψ denote the encoder, decoder, and prior network in a conditional VAE framework, respectively.In particular, ϕ is a recurrent neural network (RNN) that encodes a molecule into a latent vector z ∈ R d , and θ is also an RNN that autoregressively generates each token of SELFIES string.The prior network ψ is our proposed protein muli-modal network in which both sequence and geometric information of a target protein are encoded, and we use two multi-layer perceptions denoted as µ ψ and σ ψ to compute the mean and variance of a Gaussian distribution over the latent vector z l ∼ N (µ ψ (p), σ ψ (p)), which is the inferred latent embedding of the ligand l.All the networks are jointly optimized based on equation (11).
After training, given a target protein p, a ligand l is generated by sampling a latent vector z ∼ N (µ ψ (p), σ ψ (p)), which is fed to the decoder θ to decode into a SELFIES representation.

Conditional inference with pretrained unconditional VAE
In addition to validity, the diversity of generated sets of ligands is also an important criterion in drug discovery.While classical conditional VAE trained on protein-ligand pairs can generate novel and valid molecules, the diversity and uniqueness of these samples are relatively low due to the limited amount of available data.We address this issue by adapting the work proposed by Harvey et al [49] from the computer vision domain to diversify the latent variables.In this framework, the decoder θ of the generative model is independent with the condition p as p θ,ψ (x, z|p) = p θ (x|z)p ψ (z|p), allowing θ to re-use weights of the decoder θ * of an unconditional VAE as both have the identical architecture.We train the model to optimize the objective as: Different from equation ( 13), both q ϕ and p θ in equation ( 11) are not conditioned by the auxiliary covariate y.This allows conditional VAEs to use weights of ϕ * and θ * of a pre-trained VAE, which is trained on a diverse set of unconditional molecules.

Binding affinity prediction on PDBBind 2020 5.1.1. Experimental setup
The objective of the task is to predict the binding affinities that reflect how small molecules (ligands) bind to their target proteins.We evaluate the performance of our approach on the PDBBind v2020 [57] with time-split training and testing sets.In particular, the dataset contains 19 119 pre-processed complexes, and there are 1152 of them were discovered in 2019 or later.We follow the split procedure described by Stärk et al [17] that randomly selects 125 unique proteins and collects all new complexes containing them to create the final test set, resulting in a total of 363 samples.For the complexes discovered before 2019, those that have ligands in the test set are removed, and this gives 17 347 complexes in total.This set is respectively split into 16 379 and 968 complexes for training and validation subsets such that they share no ligands.

Implementation details
In this task, we use three layers of GVP with a hidden dimension of 128 to extract the local geometric information of proteins, while also considering their interactions with the ligands, which are encoded by a three-layer GAT network with the same dimension.Regarding the language modeling component, we extract the residue-level embeddings of the pre-trained ESM-2 proposed by [29] with a hidden dimension of 1280.The features are fused and passed to a four-layer MLP with hidden dimensions of 1024 512, and 256 for making the predictions.We use batch normalization and a dropout rate of 0.05 across layers.The models are trained in 50 epochs with a batch size of 32 and a learning rate of 0.0001.

Main results
According to table 1, our proposed approach outperforms other sequence-based methods by a large margin, demonstrating the necessity of modeling three-dimensional structures of proteins in machine learning.Compared with structure-based and complex-based methods, the model shows comparable performances across all metrics with lower standard deviations.Furthermore, the model performs on par with other more sophisticated state-of-the-art methods, namely PSICHIC [58] and TankBind.It is worth noting that PSICHIC also leverages the residue-level embeddings extracted from the pre-trained ESM; however, Koh et al [58] use these embeddings to construct 2D graphs of proteins.This does not preserve the SE(3)-symmetry (rotations and translations), which is an important property in learning three-dimensional structures.

Ablation studies
To comprehensively evaluate the effectiveness of protein multi-modal learning, we conducted an ablation study to assess the necessity of sequence information in protein modeling.Specifically, we re-trained our model without the sequence embeddings from pre-trained ESM, keeping the 3D modeling component unchanged.The numerical results in table 2 show that exploiting both sequence and geometric information can lead to superior performance in protein-ligand binding affinity prediction.The model that combines both sources of information outperforms its 3D modeling counterpart by a large margin in Pearson, Spearman, r 2 m , and CI scores.Additionally, we observe that using only ESM embeddings can also achieve comparable results to other state-of-the-art methods, indicating the importance of protein primary structure, which is the most common protein data modality.

Target-aware ligand generation 5.2.1. Experimental setup and implementation details
This task aims to generate small drug-like molecules that bind to given target proteins with unknown binding sites.We train a prior network that encodes geometric and sequence information of a target by optimizing the objective in equation (11).We use the train set of PDBBind 2020 is used, as described in section 5.1.2.For docking simulation, we adopt AutoDock Vina [13] to compute binding affinity.The score, named Vina Score, characterizes the free energy changes of binding processes in kcal mol −1 .We test our approach with nine target proteins, including G-protein coupling receptors and kinases from DUD-E [59] and the SARS-Conv-2 main protease [60].Notably, these targets are unseen to the model during the training stage.For implementation, we use the architecture and parameters of a pre-trained RNN-based VAE proposed by Gao et al [61] in which the dimensions of latent vectors are 128.For each test target, we generate 100 molecules and estimate their drug-likeness (QED), synthetic accessibility (SA), and binding affinity (Vina Score)d.The first two metrics are calculated by RDkit, and docked poses are generated by the AutoDock Vina [13] (Vina score).Additionally, we use Obabel [62] to optimize the three-dimensional atom positions of the generated molecules.

Qualitative results
Apart from pocket-based methods [14][15][16], our generative model is conditioned on the entire protein structures, meaning that the knowledge of binding sites is unknown to the model.Using protein multi-modalities, as well as modeling the long-range interactions among residues enables us to extract representations that incorporate both geometric and sequence meanings of a target protein.These informative conditions, as shown in figure 4, allow the generative model generate multiple ligands that can bind to different sites with different poses on a given target protein.Furthermore, we choose some of the generated ligands and visualize how they bind to their targets in figure 6.

Quantitative results
Figure 5 illustrates the distribution of binding affinities of the generated molecules for their corresponding receptors.We used Autodock Vina to compute the docking scores between the ligands and receptors, with lower scores indicating stronger binding.The histograms show that our method can generate ligands with high binding affinities (low docking scores).The average binding affinity for each target protein is less than −6 kcal mol −1 .Moreover, table 3 shows the average scores for binding affinities (BA), drug-likeness (QED), and synthetic accessibility (SA) of the top 1, 10, and 20 generated molecules ranked according to their binding affinities to corresponding targets.We observe that TargetVAE can generate molecules with high binding affinity while maintaining low SA scores ranging from 4.0 to 8.0.However, we also note that our   approach is limited in generating compounds with low drug-likeness (QED).We hypothesize that this is because generating ligands that simultaneously satisfy all objectives is challenging, and our model only focuses on binding affinity.This means that there may be a trade-off between the metrics.

Conclusion
This paper proposes Protein Multimodal Network (PMN), a novel neural architecture that learns to combine multiple levels of information (i.e.multimodal) of proteins including protein sequence (i.e. from the primary structure) as well as residue-level graph and geometry (i.e. from the 3D tertiary structure) into a unified representation.Our experiments in predicting protein-ligand binding affinity on the PDBBind v2020 dataset have shown that this new multimodal representation is highly effective in capturing both local and global structural information of protein, with competitive performance against current state-of-the-art methods.Furthermore, we build a conditional variational autoencoder named TargetVAE that can generate new ligands that can bind to a target protein and have desirable properties such as high binding affinity and high synthesizability, etc.It is important to note that our case, the binding sites are not known in advance; therefore, we utilize our pre trained PMN to encode the whole protein structure as a prior / condition to guide the generation process.We evaluate our generative model and the quality of generated ligands both quantitatively and qualitatively.The candidate ligands we generate have shown great potential in in-silico simulation.
We believe our two main contributions, PMN and TargetVAE, are highly applicable in practice.In general, PMN can help scientists estimate several protein functionalities and properties, as well as produce a sophisticated protein embedding that can be used for other downstream tasks.PMN in combination with TargetVAE allows us to generate new ligands for new proteins such as mutant proteins.In our future work, we plan to examine these candidates further in vitro and in vivo experiments.
linear transformations W m and W h that work on the scalar and vector features respectively, followed by nonlinearities σ and σ + .Before being transformed, the scalar feature s is concatenated with the L 2 − norm of the vector feature V.This enables GVP to extract the rotation-invariant information from the input vector V.Moreover, an additional transformation W µ is used to control the dimensionality of the output vector V ′ , making it independent the number of norms extracted.Albeit simple, GVP is an effective module that guarantees desired properties of invariance/equivariance and expressiveness.The scalar and vector outputs of GVP are invariant and equivariant respectively, with respect to an arbitrary composition R of rotations and reflections in 3D Euclidean space.In other words, if GVP(s, V) = (s ′ , V ′ ), then GVP(s, R(V)) = (s ′ , R(V ′ )).
Input: Scalar and vector features (s, V) ∈ R n × R µ×3 Output: Scalar and vector features (s

A.2. Variational auto-encoders
A variational auto-encoder (VAE) is regarded as an auto-encoding variational Bayes model [63] that comprises two components, including a generative model and an inference model (also known as probabilistic encoder).The former uses a probabilistic decoder p θ (x|z) and a prior p ψ (z) to define a joint distribution p θ,ψ (x, z) = p θ (x|z)p ψ (z) between latent variables z and data x; in addition, Kingma and Welling [63] let p ψ (z) be isotropic Gaussian.An ideal generative model should learn to maximize the log-likelihood log p θ,ψ (x) = log ´pθ,ψ (x, z)dz.However, this is intractable as marginalization over the latent space is usually infeasible with realistic data.VAE alleviates this issue by using an encoder q ϕ (z|x) to approximate the true posterior distribution of the latent space and maximize the evidence lower bound (ELBO) over each training sample x: In conditional VAE, the generative component is augmented by auxiliary covariates y.Given a condition y, the generative model defines a conditional joint distribution of z and x as p θ,ψ (x, z|y) = p θ (x|y, z)p ψ (z|y).Similarly, the condition inputs are integrated into the encoder as q ϕ (z|x, y).These two extensions establish a prominent conditional VAE model [64][65][66][67] that is trained to maximize the conditional ELBO as: log p θ,ψ (x|y) ⩾ O cond ≜ E q ϕ [log p θ,ψ (x|y, z)] − KL [q ϕ (z|x, y) ||p ψ (z|y)] . (

Figure 1 .
Figure1.Overview of our PMN.The architecture consists of two components.Top: the three-dimensional structure of the input protein is processed by a local encoder built upon a message-passing neural network (MPNN) enhanced with geometric vector perceptron module (GVP)[26].Subsequently, a Transformer processes the extracted invariant features to capture the global interactions between distant residues.Bottom: a Transformer-based language model is used to process the input protein sequence.

Figure 2 .
Figure 2. A framework for predicting binding affinities between proteins and ligands.The protein multi-modal network (PMN) computes a unified representation from the sequence and structure of the input protein.Besides, a message-passing neural network (MPNN) and feed-forward network (FFN) are used to process the 2D graph and Morgan Fingerprint of the input ligand, respectively.Finally, all the representations are combined and passed to a FFN that predicts the binding affinity score.

Figure 3 .
Figure 3. TargetVAE with an encoder, decoder, and a prior network.The PMN prior network computes the conditions from protein structures for constructing the latent space of the VAE framework, which learns to generate SELFIES representations of molecules.

Figure 4 .
Figure 4. Multiple generated ligands with different poses bind to the 1iep target.

Figure 5 .
Figure 5.The binding affinity (in kcal mol −1 ) distribution of the generated molecules on different target proteins.

Figure 6 .
Figure 6.Three-dimensional poses of protein-ligand complexes generated by AutoDock Vina and visualized by Chimerax.Each figure is associated with the name of each target protein, synthetic accessibility, and binding affinity in kcal mol −1 of the generated ligand.

Table 1 .
Performance comparison on the PDBBind v2020 dataset.An upward arrow (↑) denotes higher scores are better, and a downward arrow (↓) denotes the reverse.Average results and standard deviations (in parentheses) from five independent runs are reported.
Note: Bold highlights the best performance.

Table 2 .
Ablation study on the use of sequence embeddings and three-dimensional structures.The results are aggregated from five independent runs.
Note: Bold highlights the best performance.

Table 3 .
Quantitative results of top k = 1, 10, 20 generated molecules, which are ranked based on binding affinity (in kcal mol −1 ).The scores are averaged over k ligands.