text_summarizer.txt

End to end Text summarization Project...

Intro and git setup
project template creation
project setup and requirements installation
logging , utils and exception module
project workflows
All components Notebook experiment
All components modular code implementation
Training pipeline
prediction pipeline and user app creation
final ci/cd deployment on aws...
======================================================================================================================================================================================

first thing s first what i had understood till now is created a new git repo in my github account and included a .gitignore one liscense file and readme file and then i had cloned that repo in my local pc.

next what i had done is opened that project in vs code and created a template.py file for the project setup purpose....

so in that file included all the necessary files including for deployment also like .github/workflows etc files and wrote a python script which reads through the files and creates files.

so by running the template.py file we were able to setup our project files from scratch clean and neat and then committed the changes to github

git add.
git commit
git push...

Now we need to create a virtual env

conda create -n textsummarizer python=3.9 -y {{{command for creating a virtual env}}}


now after creating virtual env opened the requirements.txt file and whatever libs are necessary wrote them ad at the end wrote ....... -e. which sets up the 


Next in the setup.py file just wrote my username , project name as per the github repo credentials and then done...

Now we need to run the requirementest.txt file to able to install all packages needed for our project...

after successfully installing all packages then i had committed the files to github.....



========================


now what will do is setup of logging exception and utils modules for our project... these one are easy to understand once you can refer afterwards...


Next what we did is running our entire project from data collection to training and inference in jupyter notebook  and i also trained the model on train dataset but cuda error tried with different epochs will see again later.



and after then what we did is updated readme file such that i can know from where i need to start the project like end to end implementation dots should be connected ....

Next Data ingestion notebook ipynb experiment....is done  and understood the things quite well and also tried generating the artifacts folder inside research folder of the project.

And then tried that notebook implementation inside the project as same updated config file and params file after that entity file  updated and also configuration.py file updated these all are just copy paste work as already implemented all these things inside jupyter notebook file named 01_data_ingestion.ipynb file....(understood the all the things happy for that)



Now inside componenets folder created a file named data_ingestion.py file and updated the code which is already present inside  01_data_ingestion.ipynb file 

Next inside pipeline also created new file named stage_01_data_ingestion.py file and inside this file just created object named config for this object and then called those functions...



And finally inside main.py file wrote this code....


from textSummarizer.pipeline.stage_01_data_ingestion import DataIngestionTrainingPipeline
from textSummarizer.logging import logger


STAGE_NAME = "Data Ingestion stage"
try:
   logger.info(f">>>>>> stage {STAGE_NAME} started <<<<<<") 
   data_ingestion = DataIngestionTrainingPipeline()
   data_ingestion.main()
   logger.info(f">>>>>> stage {STAGE_NAME} completed <<<<<<\n\nx==========x")
except Exception as e:
        logger.exception(e)
        raise e
        
        
And now when you run the code file python main.py a new folder named aritfacts is generated with all the train test valid files .............So successfully completed data ingestion part.

Same with the data validation as well if you stick with the ipynb file of data validation then you can easily write code for data validation as well...