How can we perform different text preprocessing methods other than Text Vectorizer ? #112
ashikshafi08
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
You can do the kind of processing you're after using the TextVectorization layer if you like: https://www.tensorflow.org/api_docs/python/tf/keras/layers/experimental/preprocessing/TextVectorization
See the
standardize
parameter where you can pass in your own function.This function you can define to preprocess text in a certain way (e.g. leave the "@" symbols in if you need). You could try different experiments and see what suits you best.
Beta Was this translation helpful? Give feedback.
All reactions