Separated framework and datasets

Moved incidents into separate master data file.  Added groups and tools to it

HTML_GENERATING_CODE/AMITT_generate_github_files.ipynb

@@ -11,7 +11,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 2,
    "metadata": {
     "scrolled": true
    },
@@ -48,7 +48,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 3,
    "metadata": {
     "scrolled": true
    },
@@ -439,7 +439,7 @@
        "[140 rows x 18 columns]"
       ]
      },
-     "execution_count": 10,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }

HTML_GENERATING_CODE/generate_amitt_ttps.py

@@ -1,7 +1,8 @@
 ''' Manage AMITT metadata
 
 The AMITT github repo at https://github.com/cogsec-collaborative/AMITT serves multiple purposes:
-* Holds the master copy of AMITT (in excel file AMITT_TTPs_MASTER.xlsx)
+* Holds the master copy of AMITT (in excel file AMITT_FRAMEWORK_MASTER.xlsx)
+* Holds the master copy of AMITT data (in excel file AMITT_DATA_MASTER.xlsx)
 * Holds detailed notes on each phase, tactic, technique, incident, task and counter in
   AMITT.  These notes are markdown pages that people are free to suggest edits to, using git's 
   fork mechanisms. 
@@ -21,7 +22,7 @@ It creates this:
 
 Here are the file inputs and outputs associated with that work: 
 
-Reads 1 excel file: ../AMITT_MASTER_DATA/AMITT_TTPs_MASTER.xlsx with sheets: 
+Reads 1 excel file: ../AMITT_MASTER_DATA/AMITT_FRAMEWORKS_MASTER.xlsx with sheets: 
 * phases
 * techniques
 * tasks
@@ -81,11 +82,16 @@ from sklearn.feature_extraction.text import CountVectorizer
 class Amitt:
 
     
-    def __init__(self, infile = '../AMITT_MASTER_DATA/AMITT_TTPs_MASTER.xlsx'):
+    def __init__(self, frameworkfile = '../AMITT_MASTER_DATA/AMITT_FRAMEWORKS_MASTER.xlsx', datafile='../AMITT_MASTER_DATA/AMITT_DATA_MASTER.xlsx'):
         
         # Load metadata from file
         metadata = {}
-        xlsx = pd.ExcelFile(infile)
+        xlsx = pd.ExcelFile(frameworkfile)
+        for sheetname in xlsx.sheet_names:
+            metadata[sheetname] = xlsx.parse(sheetname)
+            metadata[sheetname].fillna('', inplace=True)
+
+        xlsx = pd.ExcelFile(datafile)
         for sheetname in xlsx.sheet_names:
             metadata[sheetname] = xlsx.parse(sheetname)
             metadata[sheetname].fillna('', inplace=True)

README.md

@@ -40,13 +40,13 @@ We love any and all suggestions for improvements, comments and offers of help -
 ## Using the Raw Data file
 
 AMITT is open source.  If you want to do your own thing with AMITT data, these will help:
-* all the master data for it is in directory [AMITT_MASTER_DATA](AMITT_MASTER_DATA). Look for the [AMITT_TTPs_MASTER.xlsx](AMITT_MASTER_DATA/AMITT_TTPs_MASTER.xlsx) spreadsheet. This contains disinformation creators' tactics, techniques, tasks, phases, and counters. 
+* all the master data for it is in directory [AMITT_MASTER_DATA](AMITT_MASTER_DATA). Look for the [AMITT_FRAMEWORKS_MASTER.xlsx](AMITT_MASTER_DATA/AMITT_FRAMEWORKS_MASTER.xlsx) spreadsheet. This contains disinformation creators' tactics, techniques, tasks, phases, and counters. 
 
 * The [AMITT TTP Guide](https://docs.google.com/document/d/1Kc0O7owFyGiYs8N8wSq17gRUPEDQsD5lLUL_3KGCgRE/edit#) has more detailed information on each technique. 
 
 * The code to create all the HTML datasheets is in directory [HTML_GENERATING_CODE](HTML_GENERATING_CODE): you'll need generate_amitt_ttps.py and all the template files. 
 
-If you have your own version of this repository and update AMITT_TTPs_MASTER.xlsx, typing "python generate_amitt_ttps.py" will update all the files above from it. 
+If you have your own version of this repository and update AMITT_FRAMEWORKS_MASTER.xlsx, typing "python generate_amitt_ttps.py" will update all the files above from it. 
 
 
 ## Who's Responsible for AMITT

detections_index.md

@@ -20,7 +20,7 @@
 <tr>
 <td><a href="detections/F00002.md">F00002</a></td>
 <td>Analyse viral fizzle</td>
-<td></td>
+<td>We have no idea what this means.  Is it something to do with the way a viral story spreads? </td>
 <td></td>
 <td>TA01 Strategic Planning</td>
 <td>D1 Detect</td>
@@ -44,7 +44,7 @@
 <tr>
 <td><a href="detections/F00005.md">F00005</a></td>
 <td>SWOT Analysis of Cognition in Various Groups</td>
-<td></td>
+<td>Strengths, Weaknesses, Opportunities, Threats analysis of groups and audience segments. </td>
 <td></td>
 <td>TA01 Strategic Planning</td>
 <td>D1 Detect</td>