summary refs log tree commit diff stats
path: root/results
diff options
context:
space:
mode:
Diffstat (limited to 'results')
-rw-r--r--results/classifier/001/README.md9
-rw-r--r--results/classifier/002/README.md9
-rw-r--r--results/classifier/003/README.md9
-rw-r--r--results/classifier/004/README.md9
-rw-r--r--results/classifier/005/README.md9
-rw-r--r--results/classifier/006/README.md9
-rw-r--r--results/classifier/007/README.md9
-rw-r--r--results/classifier/008/README.md9
-rw-r--r--results/classifier/009/README.md9
-rw-r--r--results/classifier/010/README.md10
-rw-r--r--results/classifier/011/README.md10
-rw-r--r--results/classifier/105/README.md9
-rw-r--r--results/classifier/108/README.md9
-rw-r--r--results/classifier/111/README.md10
14 files changed, 129 insertions, 0 deletions
diff --git a/results/classifier/001/README.md b/results/classifier/001/README.md
new file mode 100644
index 000000000..c77d649cc
--- /dev/null
+++ b/results/classifier/001/README.md
@@ -0,0 +1,9 @@
+## 001
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+First iteration with the labels *instruction*, *mistranslation*, *other* and *semantic*.
diff --git a/results/classifier/002/README.md b/results/classifier/002/README.md
new file mode 100644
index 000000000..1b3e12116
--- /dev/null
+++ b/results/classifier/002/README.md
@@ -0,0 +1,9 @@
+## 002
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+The category *boot* was added.
diff --git a/results/classifier/003/README.md b/results/classifier/003/README.md
new file mode 100644
index 000000000..f7d3a17f3
--- /dev/null
+++ b/results/classifier/003/README.md
@@ -0,0 +1,9 @@
+## 003
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+The categories *KVM* and *network* were added.
diff --git a/results/classifier/004/README.md b/results/classifier/004/README.md
new file mode 100644
index 000000000..446d42498
--- /dev/null
+++ b/results/classifier/004/README.md
@@ -0,0 +1,9 @@
+## 004
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+The categories *assembly*, *graphic*, *vnc* and *device* were added.
diff --git a/results/classifier/005/README.md b/results/classifier/005/README.md
new file mode 100644
index 000000000..aaf57dceb
--- /dev/null
+++ b/results/classifier/005/README.md
@@ -0,0 +1,9 @@
+## 005
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+If a category, which suggests a bug we want to discard, has a score over *0.92*, it gets applied, even if it is not the highest score.
diff --git a/results/classifier/006/README.md b/results/classifier/006/README.md
new file mode 100644
index 000000000..7f46ce3c6
--- /dev/null
+++ b/results/classifier/006/README.md
@@ -0,0 +1,9 @@
+## 006
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+Removes the categories *instruction*, *mistranslation* and *assembly*.
diff --git a/results/classifier/007/README.md b/results/classifier/007/README.md
new file mode 100644
index 000000000..ea1ac3904
--- /dev/null
+++ b/results/classifier/007/README.md
@@ -0,0 +1,9 @@
+## 007
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+Adds the categories *permissions*, *files*, *PID*, *performance* and *debug*. It also applies the category *other* if *semantic* has a score < 0.91 (This was an error, fixed in 009).
diff --git a/results/classifier/008/README.md b/results/classifier/008/README.md
new file mode 100644
index 000000000..c0867435f
--- /dev/null
+++ b/results/classifier/008/README.md
@@ -0,0 +1,9 @@
+## 008
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+Adds the categories *all* and *none*. A bug is *all*, if all categories have a score > 0.9 and *none* if all categories have a score < 0.6.
diff --git a/results/classifier/009/README.md b/results/classifier/009/README.md
new file mode 100644
index 000000000..32fd45082
--- /dev/null
+++ b/results/classifier/009/README.md
@@ -0,0 +1,9 @@
+## 009
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 89
+```
+
+Fixes 007: If *semantic* has a score < 0.91 **and** is the highest category, it is categorized as *other*.
diff --git a/results/classifier/010/README.md b/results/classifier/010/README.md
new file mode 100644
index 000000000..34746d1b2
--- /dev/null
+++ b/results/classifier/010/README.md
@@ -0,0 +1,10 @@
+## 010
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+compare = "MoritzLaurer/deberta-v3-large-zeroshot-v2.0"
+number_bugs = 89
+```
+
+Adds another model and compares both results. If the models disagree, put the bug into *review*.
diff --git a/results/classifier/011/README.md b/results/classifier/011/README.md
new file mode 100644
index 000000000..e34953d2a
--- /dev/null
+++ b/results/classifier/011/README.md
@@ -0,0 +1,10 @@
+## 011
+
+```python
+multi_label = False
+model = "facebook/bart-large-mnli"
+compare = "MoritzLaurer/deberta-v3-large-zeroshot-v2.0"
+number_bugs = 89
+```
+
+Same as 010, but sets *multi_label* to false.
diff --git a/results/classifier/105/README.md b/results/classifier/105/README.md
new file mode 100644
index 000000000..6e126259a
--- /dev/null
+++ b/results/classifier/105/README.md
@@ -0,0 +1,9 @@
+## 105
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 5812
+```
+
+Same as 005, but classifies all bugs.
diff --git a/results/classifier/108/README.md b/results/classifier/108/README.md
new file mode 100644
index 000000000..49aa53e57
--- /dev/null
+++ b/results/classifier/108/README.md
@@ -0,0 +1,9 @@
+## 108
+
+```python
+multi_label = True
+model = "facebook/bart-large-mnli"
+number_bugs = 5812
+```
+
+Same as 008, but classifies all bugs.
diff --git a/results/classifier/111/README.md b/results/classifier/111/README.md
new file mode 100644
index 000000000..a98a9a660
--- /dev/null
+++ b/results/classifier/111/README.md
@@ -0,0 +1,10 @@
+## 111
+
+```python
+multi_label = False
+model = "facebook/bart-large-mnli"
+compare = "MoritzLaurer/deberta-v3-large-zeroshot-v2.0"
+number_bugs = 1255
+```
+
+Same as 011, but classifies 1255 bugs.