Yoshops_Data_Science_Intern(Week 1 Task 6).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "17d18fd5",
   "metadata": {},
   "source": [
    "# Yoshops Data Science Intern (Week 1 Task 6)\n",
    "# Week 1 Task 6 --> Prepare python programm for data cleaning process to removing unnecessary data\n",
    "\n",
    "## <u>Name</u> : ADVAIT GURUNATH CHAVAN\n",
    "## <u>Email Id</u> : advaitchavan135@gmail.com , advaitchavan.yoshops@gmail.com\n",
    "## <u>Mobile No</u> : +91 70214 55852"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "660b8e96",
   "metadata": {},
   "source": [
    "### For this task I have used the Yoshops_Feedback dataset provided\n",
    "### The main goal/aim is to identify the type of positive and negative feedbacks given\n",
    "### For this purpose I have used the concept of sentimental analysis\n",
    "### But, few words like 'None','NA','-', 'N','NONE','NA' makes the algorithm to identify them as negative feedbacks\n",
    "### There are times when the customer does not have any feedback to give so he/she writes such words in the feedback section; and before doing the sentimental analysis it is necessary to get rid of such words so as to get the perfect count of positive and negative types of feedback given"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cff05af8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No model was supplied, defaulted to distilbert-base-uncased-finetuned-sst-2-english and revision af0f99b (https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english).\n",
      "Using a pipeline without specifying a model name and revision in production is not recommended.\n",
      "Some layers from the model checkpoint at distilbert-base-uncased-finetuned-sst-2-english were not used when initializing TFDistilBertForSequenceClassification: ['dropout_19']\n",
      "- This IS expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some layers of TFDistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased-finetuned-sst-2-english and are newly initialized: ['dropout_59']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "#importing the necessary dependencies\n",
    "import pandas as pd\n",
    "from transformers import pipeline\n",
    "sentiment_analysis_pipeline = pipeline(\"sentiment-analysis\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d49f5a5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Reading the excel file\n",
    "df = pd.read_excel('Yoshops_Feedback.xlsx')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "16ec4c80",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>11. Any IDEA or Suggestions for Yoshops Startup</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>No Answer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>No Answer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Management should be better, I Think If you wo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Marketing</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    11. Any IDEA or Suggestions for Yoshops Startup \n",
       "0                                          No Answer\n",
       "1                                          No Answer\n",
       "2  Management should be better, I Think If you wo...\n",
       "3                                                 No\n",
       "4                                          Marketing"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "694e3501",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'NA']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "3a4832b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'N/a']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a4583737",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != '-']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5ca1b72f",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Vzbz']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0a050cc8",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'N/A']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4b187548",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'NIL']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "584b08ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Na']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "7fd8a046",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Not applicable']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "73aa4248",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'H']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "8185188f",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Bo']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0d82e185",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'NONE']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d1426477",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'None']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "e1d21ac9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1363"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "00a8bbb2",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Nahi']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "1a8eb7d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1362"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "f00398bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Nononononono']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "1f684e28",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Hmmm']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "3e8a192b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Nil']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "2ebf5322",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'nil']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "e23abd4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Ni']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "2eb86477",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['11. Any IDEA or Suggestions for Yoshops Startup '] != 'Nuce']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "615ed203",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of positive type of reviews:  327\n",
      "Number of negative type of reviews:  182\n",
      "Total number of types of reviews given:  509\n"
     ]
    }
   ],
   "source": [
    "df_list = df['11. Any IDEA or Suggestions for Yoshops Startup '].value_counts().keys().to_list()\n",
    "\n",
    "pos_feedbacks = []\n",
    "neg_feedbacks = []\n",
    "\n",
    "for i in df_list:\n",
    "    data = i\n",
    "    #print(i,\"    \",sentiment_pipeline(data))\n",
    "    if sentiment_analysis_pipeline(data)[0]['label'] == 'POSITIVE':\n",
    "        pos_feedbacks.append(i)\n",
    "    else:\n",
    "        neg_feedbacks.append(i)\n",
    "print(\"Number of positive type of reviews: \", len(pos_feedbacks))\n",
    "print(\"Number of negative type of reviews: \", len(neg_feedbacks))\n",
    "print(\"Total number of types of reviews given: \", len(df_list))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "93ba5a42",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}