add some notes

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/*.png filter=lfs diff=lfs merge=lfs -text

app.py

@@ -16,6 +16,7 @@ st.markdown("Condense your Browser Tabs into a few impactful words. - Inspired i
 
 
 # Sidebar
+st.sidebar.title("🔬 Test this Model!")
 st.sidebar.caption("Tidy Tabs - Title")
 user_input_url = st.sidebar.text_input('Enter your url:')
 
@@ -34,21 +35,24 @@ def load_tab():
             else:
                 
                 with st.spinner('Wait for it...'):
-                    st.sidebar.write(f'<title>: **{title}**')
+                    st.sidebar.write(f'**<title>: **{title}')
                     time.sleep(1)
                 with st.spinner('Wait for it...'):
-                    st.sidebar.write(f'T5-small: **{predict_model_t5(text)}**')
+                    st.sidebar.write(f'**T5-small: **{predict_model_t5(text)}')
                 with st.spinner('Wait for it...'):
-                    st.sidebar.write(f'Pegasus xsum: **{predict_model_pegasus(text)}**')
+                    st.sidebar.write(f'**Pegasus xsum: **{predict_model_pegasus(text)}')
                 with st.spinner('Wait for it...'):
-                    st.sidebar.write(f'Pegasus Bart: **{predict_model_bart(text)}**')
+                    st.sidebar.write(f'**Bart-Large-Cnn: **{predict_model_bart(text)}')
         else:
             error_message = st.sidebar.error(f'{text} is not a valid URL. Please enter a valid URL.')
 
-button_clicked = st.sidebar.button("Load tab", on_click=load_tab())
+button_clicked = st.sidebar.button("Rename the tab", on_click=load_tab())
 
 st.sidebar.divider()
-
+###
+# Content
+###
+st.image('./assets/banner_tabs.png', width=350, caption='Navigate Through Powerful Features with Intuitive Tabs')
 
 with st.status("Loading models...", expanded=True, state="complete") as models:
     st.write("Loading https://huggingface.co/wgcv/tidy-tab-model-t5-small")
@@ -66,4 +70,119 @@ with st.status("Loading models...", expanded=True, state="complete") as models:
     models.update(label="All models loaded!", state="complete", expanded=False)
 
 
+st.info("All three models are deployed in a single Hugging Face Space using the free tier. Specifications: CPU-based (no GPU), 2 vCPU cores, 16 GB RAM, and 50 GB storage.",icon="ℹ️")
+###
+# Examples
+###
+
+
+st.markdown("""
+            
+Here are some examples you can try that aren't included in the training or test datasets
+# Examples
+```
+Urls:
+    https://www.nytimes.com/2007/01/10/technology/10apple.html
+    https://www.nytimes.com/2021/04/15/arts/design/Met-museum-roof-garden-da-corte.html
+    https://www.forbes.com/sites/davidphelan/2024/07/09/apple-iphone-16-pro-major-design-upgrade-coming-new-report-claims/
+    https://www.crn.com/news/channel-programs/18828789/microsoft-to-release-windows-xp-service-pack-1
+    https://github.com/torvalds
+    https://www.rickbayless.com/recipe/pastor-style-tacos/
+        
+    Some websites, like x.com, are not accessible because they use JavaScript engines to load content, which is beyond the scope of this project.
+
+```
+            
+# The Dataset
+The project's creator collected the dataset from various sources on the internet. 
+The dataset includes: 
+            
+| Feature                    | Description                                                                                   |
+|----------------------------|-----------------------------------------------------------------------------------------------|
+| URL                        | URL of the webpage                                                                            |
+| title                      | Extracted from the HTML `<title>`                                                             |
+| description                | Extracted from `<meta name="description" content="description">`                              |
+| paragraphs                 | Extracted from `<p>` tags                                                                     |
+| headings                   | Extracted from `<h1>`, `<h2>`, `<h3>` tags                                                    |
+| combined text              | Formatted as `[title] title \\n [description]description`                                     |
+
+
+The dataset primarily comprises data gathered from nytimes.com and GitHub.com, supplemented by approximately 60 other websites featuring diverse content. From GitHub, 1226 summaries were created programmatically, creating the summary with the format Username GitHub Profiles to explore the model's ability to generate patterns with new words. For the New York Times, 1056 websites were summarized based on their text content, using Claude 3.5 Sonnet of Anthropic with a specified prompt.
+## Prompt for labels Generation
+```
+        Claude 3.5 Sonnet Prompt
+             
+            I’m going to share with you a csv file with one column . I want you to create a summary of 1 to 3 words maximum of the text. The text could have HTML tags. The title is the title of the page and the description is the page's description. 
+            The result gives me like this 
+            ``` 
+            summary 1
+            summary 2
+            ...
+            summary n
+            ```
+            Only plain text and no additional instructions
+```
+
+- This small dataset aims to provide an initial assessment of model performance in a pre-trained task limited to concise summaries of 1 to 4 words. Due to the inherent complexity of this task, I suggest future efforts focus on constructing a larger dataset comprising 50,000 to 500,000 websites to more comprehensively evaluate model capabilities.
+
+- Testing revealed that the description meta tag significantly enhanced result generation. Increasing dataset size and incorporating contextual data are expected to further improve model performance in larger-scale applications with millions of data points.
+
+- Out of the 60 additional websites included, only 41 are sourced from substack.com. This means that less than 2% of the dataset contains information from substack.com. This is valuable for understanding the impact of small data examples.
+
+- P.S. I tested ChatGPT-4.0, and the results were highly discouraging for a chunk of data consisting of 100 text filed values.
+
+-In the future, we should aim to increase the dataset size to at least 10,000-15,000 samples and improve the train/test/validation split methodology.
+
+st.info("I crafted this dataset using a more powerful LLM and scripts, no need for boring manual labeling. The idea is to eliminate human labeling.",icon="ℹ️")
+
+#### Access to the data
+            
+`https://huggingface.co/datasets/wgcv/website-title-description`
+
+# Models
+My objective was to show that it was possible to create a small ML model from a bigger LLM model that could achieve good or better results in specific tasks compared to the original LLM
+
+Given the substantial volume of data, training a model from scratch was deemed impractical. Instead, our approach focused on evaluating the performance of existing pre-trained models as a baseline. This strategy served as an optimal starting point for developing a custom, lightweight model tailored to our specific use case: enhancing browser tab organization and efficiently summarizing the core concepts of favorited websites.
+
+### T5-small
+- The [T5-small](https://huggingface.co/wgcv/tidy-tab-model-t5-small) model is a finetuning of google-t5/t5-small.
+- It's a text-to-text model
+- It's a general model for all NLP tasks
+- The task is defined by the input format
+- To perform summarization, prefix the text with 'summarize:'
+- 60.5M parameters
+
+### Pegasus-xsum
+- The [Pegasus-xsum](https://huggingface.co/wgcv/tidy-tab-model-pegasus-xsum) model is a finetuning of google/pegasus-xsum.
+- It's a text-to-text model
+- It's a specialized summarization model
+- 570M params
+
+### Bart-large
+- The [Bart-large](https://huggingface.co/wgcv/tidy-tab-model-bart-large-cnn) model is a finetuning of facebook/bart-large-cnn.
+- Prior to our fine-tuning, it was fine-tuned on the CNN/Daily Mail dataset.
+- It's a BART model, using a transformer encoder-decoder (seq2seq) architecture.
+- BART models typically perform better with small datasets compared to text-to-text models.
+- 406M params
+
+
+            
+### Potential avenues for performance enhancement include:
+- Data preprocessing optimization
+- Dataset expansion
+- Comprehensive hyperparameter tuning
+- These strategies could significantly improve model efficacy.
+
+## co2_eq_emissions
+- emissions: 0.16 grams of CO2)
+- source: mlco2.github.io
+- training_type: fine-tuning
+- geographical_location: U.S.
+-  hardware_used: 1 - T4 GPU
+        
+## 
+
+
+
+""", unsafe_allow_html=False, help=None)