Classification¶
-
class
Manteia.Classification.
Classification
(model=None, documents_train=[], labels_train=[], documents_test=[], labels_test=[], process_classif=False, verbose=True)¶ This is the class to classify text in categorie a NLP task.
Args:
- model_name (
string
, optional, defaults to ‘bert’): give the name of a model.
- documents (
list
, optional, defaults to None): A list of documents.
- labels (
float
, optional, defaults to None): A list of labels.
Example 1:
from Manteia.Classification import Classification from Manteia.Model import Model documents = ['What should you do before criticizing Pac-Man? WAKA WAKA WAKA mile in his shoe.' ,'What did Arnold Schwarzenegger say at the abortion clinic? Hasta last vista, baby.',] labels = ['funny','not funny'] model = Model(model_name ='roberta') cl=Classification(model,documents,labels,process_classif=True) >>>Training complete!
-
load_model
()¶ Example 3:
from Manteia.Classification import Classification from Manteia.Preprocess import list_labels documents = ['What should you do before criticizing Pac-Man? WAKA WAKA WAKA mile in his shoe.' ,'What did Arnold Schwarzenegger say at the abortion clinic? Hasta last vista, baby.',] labels = ['funny','not funny'] cl=Classification(documents_train = documents,labels_train = labels) cl.list_labels = list_labels(labels) cl.load_model() cl.model.devices() print(cl.predict(documents[:2])) >>>['funny', 'funny']
-
predict
(documents)¶ This is the description of the predict function.
Args:
- documents (
list
, optional, defaults to None): A list of documents (str).
Example 5:
from Manteia.Classification import Classification from Manteia.Model import Model documents = ['What should you do before criticizing Pac-Man? WAKA WAKA WAKA mile in his shoe.' ,'What did Arnold Schwarzenegger say at the abortion clinic? Hasta last vista, baby.',] labels = ['funny','not funny'] model = Model(model_name ='roberta') cl=Classification(model,documents,labels,process_classif=True) print(cl.predict(documents[:2])) >>>['funny', 'funny']
- documents (
-
process
()¶ Example 2:
from Manteia.Classification import Classification from Manteia.Preprocess import list_labels from Manteia.Model import Model documents = ['What should you do before criticizing Pac-Man? WAKA WAKA WAKA mile in his shoe.' ,'What did Arnold Schwarzenegger say at the abortion clinic? Hasta last vista, baby.',] labels = ['funny','not funny'] model = Model(model_name ='roberta') cl=Classification(model,documents,labels) cl.list_labels = list_labels(labels) cl.process() print(cl.predict(documents[:2])) >>>['funny', 'funny']
-
process_text
()¶ This is the description of the process_text function.
Example 4:
from Manteia.Classification import Classification from Manteia.Preprocess import list_labels documents = ['What should you do before criticizing Pac-Man? WAKA WAKA WAKA mile in his shoe.' ,'What did Arnold Schwarzenegger say at the abortion clinic? Hasta last vista, baby.',] labels = ['funny','not funny'] cl=Classification(documents_train = documents,labels_train = labels) cl.list_labels = list_labels(labels) cl.load_model() dt_train ,dt_validation=cl.process_text() cl.model.configuration(dt_train) cl.model.fit(dt_train,dt_validation) >>>Training complete!
- model_name (
A complete example¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | from Manteia.Classification import Classification
from Manteia.Preprocess import Preprocess
from Manteia.Dataset import Dataset
def main(args):
ds = Dataset('20newsgroups')
documents = ds.documents_train
labels = ds.labels_train
pp = Preprocess(documents=documents,labels=labels,nb_sample=500)
documents = pp.documents
labels = pp.labels
cl = Classification(documents_train=documents,labels_train=labels)
cl.list_labels = pp.list_labels
cl.load_model()
dt_train ,dt_validation = cl.process_text()
cl.model.configuration(dt_train)
cl.model.fit(dt_train,dt_validation)
print(cl.predict(documents[:5]))
return 0
if __name__ == '__main__':
import sys
sys.exit(main(sys.argv))
|