Spark NLP - Model Serving Databricks

[Rajora0]

I encountered this issue within Databricks. Is there any way to solve this? Has anyone done something similar? I found some potentially relevant information at the following link: https://www.johnsnowlabs.com/serving-spark-nlp-via-api-3-3-databricks-jobs-and-mlflow-serve-apis/

In the link, I found the following answer:

IMPORTANT: As of 17/02/2022, there is an issue being studied by the Databricks team regarding the creation of job clusters on the fly to serve MLFlow models that require configuring the Spark Session with specific jars. This will be fixed in later versions of Databricks. In the meantime, the way to proceed is by using the Databricks Jobs API.