3D-e-Chem · sverhoeven · May 2, 2017 · Apr 21, 2017 · Apr 24, 2017 · Apr 25, 2017
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -11,6 +11,13 @@ Formatted as described on http://keepachangelog.com/.
 - Similarities histogram can read frozen matrices using either lower or upper triangle
 - Similarities export can be filtered by frag1 and/or pdb codes
 - Use scripts in update steps
+- Pharmacophores
+  - Store pharmocophore points in pytables table (#29)
+  - Export pharmacophore points in [\*.phar format](http://silicos-it.be.s3-website-eu-west-1.amazonaws.com/software/align-it/1.0.4/align-it.html#format)
+  - Sub command to add points to table from a directory
+  - Sub command to filter the pharmacophores points based on a fragments database
+  - Sub command and webservice endpoint to fetch the pharmacophore points of a single fragment identifier (#30)
+  - Canned method to fetch pharmacophores of a list fragment identifiers
 
 ### Fixed
 

diff --git a/README.md b/README.md
@@ -44,23 +44,7 @@ kripodb --help
 
 ## Create all
 
-Commands to create all data files
-```
-kripodb fragments shelve fragments.shelve fragments.sqlite
-kripodb fragments sdf fragment??.sdf fragments.sqlite
-kripodb fragments pdb fragments.sqlite
-kripodb fingerprints import 01.fp 01.fp.db
-kripodb fingerprints import 02.fp 02.fp.db
-kripodb fingerprints similarities --fragmentsdbfn fragments.sqlite --ignore_upper_triangle 01.fp.db 01.fp.db sim_01_01.h5
-kripodb fingerprints similarities --fragmentsdbfn fragments.sqlite --ignore_upper_triangle 02.fp.db 02.fp.db sim_02_02.h5
-kripodb fingerprints similarities --fragmentsdbfn fragments.sqlite 01.fp.db 02.fp.db sim_01_02.h5
-kripodb similarities merge sim_*_*.h5  sim_all.h5
-kripodb similarities freeze sim_all.h5 sim_all.frozen.h5
-# Make froze similarity matrix smaller, by using slower compression
-ptrepack --complevel 6 --complib blosc:zlib sim_all.frozen.h5 sim_all.packedfrozen.h5
-rm sim_all.frozen.h5
-kripodb similarities serve sim_all.packedfrozen.h5
-```
+Commands to create all data files see [update documentation](docs/data-update.rst).
 
 ## Search for most similar fragments
 
@@ -71,22 +55,7 @@ kripodb similar sim_all.h5 3kxm_K74_frag1 --cutoff 0.45
 
 ## Create similarity matrix from text files
 
-Input files `sim_??_??.txt.gz` looks like:
-```
-Compounds similar to 2xry_FAD_frag4:
-2xry_FAD_frag4   1.0000
-3cvv_FAD_frag3   0.5600
-```
-
-To create a single similarity matrix from multiple text files:
-```
-gunzip -c sim_01_01.txt.gz | kripodb similarities import --ignore_upper_triangle - fragments.sqlite sim_01_01.h5
-gunzip -c sim_01_02.txt.gz | kripodb similarities import - fragments.sqlite sim_01_02.h5
-gunzip -c sim_02_02.txt.gz | kripodb similarities import --ignore_upper_triangle - fragments.sqlite sim_02_02.h5
-kripodb similarities merge sim_??_??.h5 sim_all.h5
-```
-
-The `--ignore_upper_triangle` flag is used to prevent scores corruption when freezing similarity matrix.
+Commands to create similarity matrix see [update documentation](docs/data-update.rst).
 
 # Data sets
 
@@ -113,13 +82,13 @@ All fragments form all proteins-ligand complexes in PDB compared with all.
 
 A data set with PDB entries till 23 December 2015 has been published at [![DOI](https://zenodo.org/badge/doi/10.5281/zenodo.55254.svg)](http://dx.doi.org/10.5281/zenodo.55254)
 
-# Knime
+# KNIME
 
-The [Knime-KripoDB-example.zip](https://github.com/3D-e-Chem/knime-kripodb/blob/master/examples/Knime-KripoDB-example.zip) file is an example workflow showing how to use KripoDB python package inside Knime (http://www.knime.org).
-It can be run by importing it into Knime.
-Make sure the Python used by Knime is the same as the Python with kripodb package installed.
+The [Knime-KripoDB-example.zip](https://github.com/3D-e-Chem/knime-kripodb/blob/master/examples/Knime-KripoDB-example.zip) file is an example workflow showing how to use KripoDB python package inside KNIME (http://www.knime.org).
+It can be run by importing it into KNIME.
+Make sure the Python used by KNIME is the same as the Python with kripodb package installed.
 
-The https://github.com/3D-e-Chem/knime-kripodb repo adds KripoDB code templates to Knime.
+The https://github.com/3D-e-Chem/knime-kripodb repo adds KripoDB code templates to KNIME.
 
 # Development of KripoDB
 
@@ -154,7 +123,7 @@ The Kripo data files can be queried using a web service.
 
 Start webservice with:
 ```
-kripodb serve data/similarities.h5 data/fragments.sqlite
+kripodb serve data/similarities.h5 data/fragments.sqlite data/pharmacophores.h5
 ```
 It will print the urls for the swagger spec and UI.
 

diff --git a/data/pharmacophores.h5 b/data/pharmacophores.h5
diff --git a/docs/api.rst b/docs/api.rst
@@ -11,6 +11,7 @@ API
     kripodb.makebits <makebits.rst>
     kripodb.modifiedtanimoto <modifiedtanimoto.rst>
     kripodb.pairs <pairs.rst>
+    kripodb.pharmacophores <pharmacophores.rst>
     kripodb.pdb <pdb.rst>
     kripodb.script <script.rst>
     kripodb.webservice <webservice.rst>

diff --git a/docs/baseline-update.rst b/docs/baseline-update.rst
@@ -48,6 +48,16 @@ The fragment molblocks can be generated into a fragment sdf file with::
 
   fragid2sd.py > fragments.sd
 
+3. Pharmacophores
+^^^^^^^^^^^^^^^^^
+
+The raw pharmacophores are stored in the FRAGMENT_PPHORES sub-directory.
+Each pocket has a \*_pphore.sd.gz file which contains the pharmacophore points of the whole pocket and
+a \*_pphores.txt file which contains the indexes of pharmacophore points for each sub pocket or fragment.
+The raw pharmacophores need to be added to the pharmacophores datafile with::
+
+    kripodb pharmacophores add FRAGMENT_PPHORES pharmacophores.h5
+
 4. Add new fragment information to fragment sqlite db
 -----------------------------------------------------
 

diff --git a/docs/incremental-update.rst b/docs/incremental-update.rst
@@ -45,6 +45,18 @@ The fragment molblocks can be generated into a fragment sdf file with::
 
   fragid2sd.py > fragments.sd
 
+
+3. Pharmacophores
+^^^^^^^^^^^^^^^^^
+
+The raw pharmacophores are stored in the FRAGMENT_PPHORES sub-directory.
+Each pocket has a \*_pphore.sd.gz file which contains the pharmacophore points of the whole pocket and
+a \*_pphores.txt file which contains the indexes of pharmacophore points for each sub pocket or fragment.
+The raw pharmacophores of the update can be added to the existing pharmacophores datafile with::
+
+    cp ../current/pharmacophores.h5 .
+    kripodb pharmacophores add FRAGMENT_PPHORES pharmacophores.h5
+
 4. Add new fragment information to fragment sqlite db
 -----------------------------------------------------
 

diff --git a/docs/pdb.rst b/docs/pdb.rst
@@ -1,5 +1,5 @@
 kripodb.pdb
-=============
+===========
 
 .. automodule:: kripodb.pdb
     :members:
diff --git a/docs/pharmacophores.rst b/docs/pharmacophores.rst
@@ -0,0 +1,5 @@
+kripodb.pharmacophores
+======================
+
+.. automodule:: kripodb.pharmacophores
+    :members:
diff --git a/kripodb/canned.py b/kripodb/canned.py
@@ -24,7 +24,8 @@
 
 from .db import FragmentsDb
 from .pairs import similar, open_similarity_matrix
-from .webservice.client import WebserviceClient, IncompleteFragments
+from .pharmacophores import PharmacophoresDb, as_phar
+from .webservice.client import WebserviceClient, IncompleteFragments, IncompletePharmacophores
 
 
 class IncompleteHits(Exception):
@@ -62,7 +63,8 @@ def similarities(queries, similarity_matrix_filename_or_url, cutoff, limit=1000)
         11
 
         Retrieved from web service instead of local similarity matrix file.
-        Make sure the web service is running, for example by `kripodb serve data/similarities.h5 data/fragments.sqlite`.
+        Make sure the web service is running,
+        for example by `kripodb serve data/similarities.h5 data/fragments.sqlite data/pharmacophores.h5`.
 
         >>> hits = similarities(queries, 'http://localhost:8084/kripo', 0.55)
         >>> len(hits)
@@ -134,7 +136,8 @@ def fragments_by_pdb_codes(pdb_codes, fragments_db_filename_or_url, prefix=''):
         3
 
         Retrieved from web service instead of local fragments db file.
-        Make sure the web service is running, for example by `kripodb serve data/similarities.h5 data/fragments.sqlite`.
+        Make sure the web service is running,
+        for example by `kripodb serve data/similarities.h5 data/fragments.sqlite data/pharmacophores.h5`.
 
         >>> fragments = fragments_by_pdb_codes(pdb_codes, 'http://localhost:8084/kripo')
         >>> len(fragments)
@@ -192,7 +195,8 @@ def fragments_by_id(fragment_ids, fragments_db_filename_or_url, prefix=''):
         1
 
         Retrieved from web service instead of local fragments db file.
-        Make sure the web service is running, for example by `kripodb serve data/similarities.h5 data/fragments.sqlite`.
+        Make sure the web service is running,
+        for example by `kripodb serve data/similarities.h5 data/fragments.sqlite data/pharmacophores.h5`.
 
         >>> fragments = fragments_by_id(fragment_ids,, 'http://localhost:8084/kripo')
         >>> len(fragments)
@@ -229,3 +233,56 @@ def fragments_by_id(fragment_ids, fragments_db_filename_or_url, prefix=''):
     df = pd.DataFrame(fragments)
     df.rename(columns=lambda x: prefix + x, inplace=True)
     return df
+
+
+def pharmacophores_by_id(fragment_ids, pharmacophores_db_filename_or_url):
+    """Fetch pharmacophore points by fragment identifiers
+
+    Args:
+        fragment_ids (pd.Series): List of fragment identifiers
+        pharmacophores_db_filename_or_url: Filename of pharmacophores db or base url of kripodb webservice
+
+    Returns:
+        pandas.Series: Pandas series with pharmacophores as string in phar format.
+                       Fragment without pharmacophore will return None
+
+    Examples:
+        Fragments similar to '3j7u_NDP_frag24' fragment.
+
+        >>> from kripodb.canned import pharmacophores_by_id
+        >>> fragment_ids = pd.Series(['2n2k_MTN_frag1'])
+        >>> pharmacophores = pharmacophores_by_id(fragment_ids, 'data/pharmacophores.h5')
+        >>> len(pharmacophores)
+        1
+
+        Retrieved from web service instead of local pharmacophores db file.
+        Make sure the web service is running,
+        for example by `kripodb serve data/similarities.h5 data/fragments.sqlite data/pharmacophores.h5`.
+
+        >>> pharmacophores = pharmacophores_by_id(fragment_ids,, 'http://localhost:8084/kripo')
+        >>> len(pharmacophores)
+        1
+    """
+    if pharmacophores_db_filename_or_url.startswith('http'):
+        client = WebserviceClient(pharmacophores_db_filename_or_url)
+        try:
+            pphors = client.pharmacophores(fragment_ids)
+        except IncompletePharmacophores as e:
+            s = pd.Series(e.pharmacophores, dtype=str)
+            raise IncompletePharmacophores(e.absent_identifiers, s)
+    else:
+        with PharmacophoresDb(pharmacophores_db_filename_or_url) as pharmacophoresdb:
+            pphors = []
+            absent_identifiers = []
+            for frag_id in fragment_ids:
+                try:
+                    phar = as_phar(frag_id, pharmacophoresdb[frag_id])
+                    pphors.append(phar)
+                except KeyError:
+                    pphors.append(None)
+                    absent_identifiers.append(frag_id)
+            if absent_identifiers:
+                s = pd.Series(pphors, dtype=str)
+                raise IncompletePharmacophores(absent_identifiers, s)
+    s = pd.Series(pphors, dtype=str)
+    return s