fix broken links

Author: nsheff

docs/bedbase/README.md

@@ -27,6 +27,5 @@ BEDbase is a unifying platform for aggregating, analyzing and serving genomic re
 - [bbconf](http://github.com/databio/bbconf): BEDbase configuration package (core of the BEDbase stack)
 - [bedhost](http://github.com/databio/bedhost): FastAPI application with API for accessing data
 - [bedhost-ui](http://github.com/databio/bedhost): Front-end user interface built with React
-- [bedbase.org repository](https://github.com/databio/bedbase.org): Repository for deploying the bedhost container to AWS.
 - `all_geo_beds`: A subfolder containing scripts to download all bed files on GEO using geofetch and build a backend to host the metadata using bedstat
 - [geniml](https://github.com/databio/geniml): Machine learning for genomic intervals

docs/bedboss/README.md

@@ -118,7 +118,7 @@ Example bedsets:
 
 - Bed files from the AML database.
 - Bed files from the [Excluderanges](https://github.com/dozmorovlab/excluderanges#bedbase-data-download) database.
-- Bed files from the LOLA database [http://lolaweb.databio.org/](http://lolaweb.databio.org/)
+- Bed files from the LOLA database [https://databio.org/regiondb](https://databio.org/regiondb)
 
 \*This pipeline is available only in for bedbase processing, and can't be use as a standalone tool.
 

docs/bedboss/tutorials/bedms_tutorial.md

@@ -75,7 +75,7 @@ To develop training sets, follow the step by step protocol mentioned below:
         values_1000.csv
     ```
         To see an example of what a `values_*.csv` and `headers_*.csv` might look like, you can check our sample csv files on PEPhub: [sample_bedms_values_1.csv](https://pephub.databio.org/databio/sample_bedms_values_1?tag=default) and [sample_bedms-headers_1.csv](https://pephub.databio.org/databio/sample_bedms_headers_1?tag=default).
-    While these are only samples and are not information dense, we recommend having large vocabulary for the training files for both the `values_directory` and `headers_directory`. To get a better understanding of the training data that we trained BEDMS on, you can visit this [link](https://big.databio.org/bedms/)
+    While these are only samples and are not information dense, we recommend having large vocabulary for the training files for both the `values_directory` and `headers_directory`. To get a better understanding of the training data that we trained BEDMS on, you can visit this [link](http://big.databio.org/bedms/)
 
 3. Once your training sets are ready, you can make a directory for your schema in your HuggingFace repository. If the name of your schema is `new_schema` and the name of your repository is `new_repo`, this is what the directory structure will look like:
     ```

docs/bedhost/README.md

@@ -9,7 +9,7 @@ It needs a path to the *bedbase configuration file*, which can be provided eithe
 
 ## Introduction
 
-You can find the formal OpenAPI documentation and interactive interface at <http://api.bedbase.org/docs>. This document provides more conceptual introduction and explanations to how to use the API effectively.
+You can find the formal OpenAPI documentation and interactive interface at <https://api.bedbase.org/v1/docs>. This document provides more conceptual introduction and explanations to how to use the API effectively.
 
 
 ## General API organization

docs/geniml/changelog.md

@@ -60,8 +60,8 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 ## [0.4.0] -- 2024-06-04
 
-- Added bed tokens caching to bbclient [bbclient] Add tokenized file cache and download [#153](https://github.com/databio/geniml_dev/issues/153)
-- Added pyBiocFileCache for bedfiles to support R caching [bbclient] Integrate bedbase caching with R [#151](https://github.com/databio/geniml_dev/issues/151)
+- Added bed tokens caching to bbclient [bbclient] Add tokenized file cache and download #153
+- Added pyBiocFileCache for bedfiles to support R caching [bbclient] Integrate bedbase caching with R #151
 - Added support of Python3.12
 - Optimized encoding of regions for Region2Vec models
 - Added updates to the new Atacformer

docs/geniml/notebooks/assess-universe.ipynb

@@ -3,13 +3,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": [
-    "# How to assess universe fit to collection of BED files\n",
-    "\n",
-    "## Introduction\n",
-    "\n",
-    "In this tutorial, you will see how to assess a fit of a given universe to a collection of files. (Tutorial on creating different universes from files can be found [here](../tutorials/create-consensus-peaks.md) and [here](create-consensus-peaks-python.md).) Choosing, which universe represents data the best can be challenging. To help with this decision we created three different metrics for assessing universe fit to the region collections: a base-level overlap score, a region boundary score, and a likelihood score. Fit of a universe can be assessed both using CLI and python functions depending on use case. With CLI you can create a file with values of universe assessment methods for each file within the collection, while with python functions you can get measures of universe fit to the whole collection. "
-   ]
+   "source": "# How to assess universe fit to collection of BED files\n\n## Introduction\n\nIn this tutorial, you will see how to assess a fit of a given universe to a collection of files. (Tutorial on creating different universes from files can be found [here](../tutorials/create-consensus-peaks/) and [here](create-consensus-peaks-python/).) Choosing, which universe represents data the best can be challenging. To help with this decision we created three different metrics for assessing universe fit to the region collections: a base-level overlap score, a region boundary score, and a likelihood score. Fit of a universe can be assessed both using CLI and python functions depending on use case. With CLI you can create a file with values of universe assessment methods for each file within the collection, while with python functions you can get measures of universe fit to the whole collection. "
   },
   {
    "cell_type": "markdown",
@@ -387,4 +381,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 4
-}
+}

docs/geniml/notebooks/create-consensus-peaks-python.ipynb

@@ -4,27 +4,7 @@
    "cell_type": "markdown",
    "id": "66137bf9-4f78-4b89-aa64-6734400df15e",
    "metadata": {},
-   "source": [
-    "# How to build a new universe?\n",
-    "\n",
-    "## Data preprocessing\n",
-    "This is a jupyter version of CLI tutorial that can be found [here](../tutorials/create-consensus-peaks.md). You will use here python functions instead of CLI to build and assess different universes. Files that you will use here can be downloaded from XXX. In there you will find a compressed folder:\n",
-    "\n",
-    "```\n",
-    "consensus:\n",
-    "    - raw\n",
-    "        test_1.bed\n",
-    "        test_2.bed\n",
-    "        test_3.bed\n",
-    "        test_4.bed\n",
-    "    file_list.txt\n",
-    "    chrom.sizes\n",
-    "```\n",
-    "\n",
-    "In the raw folder there are example BED files used in this tutorial and in file_list.txt are names of files you will analyze. Additionally there is a file with chromosome sizes, which you will use to preprocess the data. \n",
-    "\n",
-    "Here we assume that you already have files of the genome coverage by the analyzed collection. The example of how to create them can be found [here](../tutorials/create-consensus-peaks.md)."
-   ]
+   "source": "# How to build a new universe?\n\n## Data preprocessing\nThis is a jupyter version of CLI tutorial that can be found [here](../tutorials/create-consensus-peaks/). You will use here python functions instead of CLI to build and assess different universes. Files that you will use here can be downloaded from XXX. In there you will find a compressed folder:\n\n```\nconsensus:\n    - raw\n        test_1.bed\n        test_2.bed\n        test_3.bed\n        test_4.bed\n    file_list.txt\n    chrom.sizes\n```\n\nIn the raw folder there are example BED files used in this tutorial and in file_list.txt are names of files you will analyze. Additionally there is a file with chromosome sizes, which you will use to preprocess the data. \n\nHere we assume that you already have files of the genome coverage by the analyzed collection. The example of how to create them can be found [here](../tutorials/create-consensus-peaks/)."
   },
   {
    "cell_type": "markdown",
@@ -170,11 +150,7 @@
    "cell_type": "markdown",
    "id": "bab159ce-aa6f-4b56-9696-59f7db7c81b2",
    "metadata": {},
-   "source": [
-    "# How to assess new universe?\n",
-    "\n",
-    "So far you used many different methods for creating new universes. But choosing, which universe represents data the best can be challenging. To help with this we created a tutorial that can be found [here](../code/assess-universe.md), which presents different  methods that assess universe fit to the collection of files."
-   ]
+   "source": "# How to assess new universe?\n\nSo far you used many different methods for creating new universes. But choosing, which universe represents data the best can be challenging. To help with this we created a tutorial that can be found [here](../tutorials/assess-universe/), which presents different  methods that assess universe fit to the collection of files."
   }
  ],
  "metadata": {
@@ -198,4 +174,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}

docs/geniml/tutorials/bedshift-evaluation-guide.md

@@ -28,7 +28,7 @@ You can repeat this for each of the similarity scores and each of the perturbati
 
 ## Using a PEP to quickly submit multiple bedshift jobs
 
-Using a [Portable Encapsulated Project](http://pep.databio.org/en/latest/) (PEP), creating multiple combinations of bedshift files becomes faster and more organized. The PEP consists of a sample table containing the perturbation parameters and a config file. Here is what the `sample_table.csv` may look like. Each row specifies the arguments for a bedshift command.
+Using a [Portable Encapsulated Project](https://pep.databio.org/) (PEP), creating multiple combinations of bedshift files becomes faster and more organized. The PEP consists of a sample table containing the perturbation parameters and a config file. Here is what the `sample_table.csv` may look like. Each row specifies the arguments for a bedshift command.
 
 | sample_name | add | drop | shift | cut | merge |
 |-------------|-----|------|------|------|-------|

docs/geniml/tutorials/bedshift.md

@@ -48,7 +48,7 @@ If you're looking to use Bedshift in your own experiment, we created an [example
 2. Run a pipeline on the dataset and obtain results
 3. Aggregate and visualize the results
 
-It integrates the [PEP](http://pep.databio.org/en/latest/) and [looper](http://looper.databio.org/en/latest/) workflow allowing you to easily
+It integrates the [PEP](https://pep.databio.org/) and [looper](https://pep.databio.org/looper/) workflow allowing you to easily
 run the project out of the box.
 
 ## Generate a random BED file

docs/geniml/tutorials/train-region2vec.md

@@ -4,7 +4,7 @@ Region2Vec is an unsupervised method for creating embeddings of genomic regions
 ## Training data and universe
 Training a model requires two things: 1) a set of pre-tokenized data and 2) a universe. The universe is a set of regions that the model will be trained on. The universe is used to create the tokenizer, which is used to convert the raw data into tokens. The universe should be representative of the data that you will be training the model on. For example, if you are training a model on human data, you should use a universe that contains human regions. If you dont have a universe, a safe bet is to use the 1000 tiles hg38 genome.
 
-You can download the 1000 tiles hg38 genome [here](https://big.databio.org/geniml/universes/tiles1000.hg38.bed).
+You can download the 1000 tiles hg38 genome [here](http://big.databio.org/geniml/universes/tiles1000.hg38.bed).
 
 The pre-tokenized data is a set of `.gtok` files. These are binary files that contain the tokenized data in the form of integers. The `.gtok` files are used directly to train the model. If you have not pre-tokenized your data, see the [pre-tokenization documentation](./pre-tokenization.md).