Giskard Scan Results

2 issues detected

Performance 1

Robustness 1

We found some data slices in your dataset on which your model performance is lower than average. Performance bias may happen for different reasons:

Not enough examples in the low-performing data slice in the training set
Wrong labels in the training set in the low-performing data slice
Drift between your training set and test set

To learn more about causes and solutions, check our guide on performance bias.

Issues

1 medium

`text` contains "know" Precision = 0.876 (Global = 0.936) -6.45% than global 113 samples affected
(5.7% of dataset) Show details Hide details

Description

For records in the dataset where `text` contains "know", the Precision is 6.45% lower than the global Precision.

Examples

	text	label	Predicted `label`
17	i know what it feels like he stressed glaring down at her as she squeezed more soap onto her sponge	anger	sadness (p = 0.89)
91	i feel like the people i know are really generous and i have my needs met	joy	love (p = 0.68)
164	i have stayed at heritage christian because of the fulfillment that i feel in doing christ s work in action by being the hands the eyes the legs and the voice of supporting the individuals that i have been blessed to know and support	joy	love (p = 0.61)

Taxonomy

avid-effect:performance:P0204

Your model seems to be sensitive to small perturbations in the input data. These perturbations can include adding typos, changing word order, or turning text into uppercase or lowercase. This happens when:

There is not enough diversity in the training data
Overreliance on spurious correlations like the presence of specific word
Use of complex models with large number of parameters that tend to overfit the training data

To learn more about causes and solutions, check our guide on robustness issues.

Issues

1 major

Feature `text`

Add typos

Fail rate = 0.213

213/1000 tested samples (21.3%) changed prediction after perturbation

1000 samples affected
(50.0% of dataset)

Show details

Description

When feature “text” is perturbed with the transformation “Add typos”, the model changes its prediction in 21.3% of the cases. We expected the predictions not to be affected by this transformation.

Examples

	text	Add typos(text)	Original prediction	Prediction after perturbation
656	i feel a little bit more nostalgic when those memories come to mind	i feel a little bit more nosftalic when those memories comwe to mind	love (p = 0.99)	joy (p = 0.94)
734	i can talk to her about almost anything i want to and she just listens and she doesnt make me feel like a whiney brat and she helps me sort my thoughts and make decisions while keeping me where she feels im safe	i can talk to her about almost anything i want to and she just lisrens and she doesnt make me feel liek a shiney brat and she helps me sort my thoughts and make decisions while keeping me where she fes im safe	sadness (p = 0.99)	joy (p = 0.99)
1403	i feel the need to preface this by saying that i am strongly in favor of keeping violent or otherwise inappropriate videogames out of the hands of minors and i believe that this is an issue that parents and the government need to work on together	i feel the need to preface this by saying that i am ateongly in faor of keeping volent or otherwise inappropriate videogames outo f yhe hands of minor san di believe that this is an issue that parents and the government need to work on tovether	anger (p = 0.97)	sadness (p = 0.88)

Taxonomy

avid-effect:performance:P0201

Debug your issues in the Giskard hub

Install the Giskard hub app to:

Debug and diagnose your scan issues
Save your scan result as a re-executable test suite to benchmark your model
Extend your test suite with our catalog of ready-to-use tests

You can find installation instructions here.

from giskard import GiskardClient

# Create a test suite from your scan results
test_suite = results.generate_test_suite("My first test suite")

# Upload your test suite to your Giskard hub instance
client = GiskardClient("http://localhost:19000", "GISKARD_API_KEY")
client.create_project("my_project_id", "my_project_name")
test_suite.upload(client, "my_project_id")