def frameIt( u:String, h:Int ) : String = { """<iframe src=""""+ u+"""" width="95%" height="""" + h + """" sandbox> <p> <a href="http://spark.apache.org/docs/latest/index.html"> Fallback link for browsers that, unlikely, don't support frames </a> </p> </iframe>""" } displayHTML(frameIt("https://en.wikipedia.org/wiki/Natural_language_processing",550))
Last refresh: Never
%md ## SparkML What does SparkML Pipelines already provide for NLP? - http://spark.apache.org/docs/latest/ml-features.html - http://spark.apache.org/docs/latest/ml-features.html#extracting-transforming-and-selecting-features - http://spark.apache.org/docs/latest/ml-features.html#word2vec - ... - https://spark.apache.org/docs/latest/mllib-feature-extraction.html - https://spark.apache.org/docs/latest/mllib-feature-extraction.html#word2vec - ... ## SparkNLP How does John Snow Labs' [sparkNLP](https://nlp.johnsnowlabs.com/) build on SparkML? - https://databricks.com/blog/2017/10/19/introducing-natural-language-processing-library-apache-spark.html #### Why SparkNLP? - https://www.oreilly.com/ideas/comparing-production-grade-nlp-libraries-training-spark-nlp-and-spacy-pipelines - https://blog.dominodatalab.com/comparing-the-functionality-of-open-source-natural-language-processing-libraries/ ### SparkNLP Pointers - https://nlp.johnsnowlabs.com/docs/en/quickstart - https://github.com/JohnSnowLabs/spark-nlp - The databricks notebooks in this directory are minor augnmentations/updates of: - https://johnsnowlabs.github.io/spark-nlp-workshop/databricks/index.html - https://nlp.johnsnowlabs.com/docs/en/models
SparkML
What does SparkML Pipelines already provide for NLP?
- http://spark.apache.org/docs/latest/ml-features.html
- http://spark.apache.org/docs/latest/ml-features.html#extracting-transforming-and-selecting-features
- https://spark.apache.org/docs/latest/mllib-feature-extraction.html
SparkNLP
How does John Snow Labs' sparkNLP build on SparkML?
Why SparkNLP?
- https://www.oreilly.com/ideas/comparing-production-grade-nlp-libraries-training-spark-nlp-and-spacy-pipelines
- https://blog.dominodatalab.com/comparing-the-functionality-of-open-source-natural-language-processing-libraries/
SparkNLP Pointers
- https://nlp.johnsnowlabs.com/docs/en/quickstart
- https://github.com/JohnSnowLabs/spark-nlp
- The databricks notebooks in this directory are minor augnmentations/updates of:
- https://nlp.johnsnowlabs.com/docs/en/models
Last refresh: Never
displayHTML(frameIt("https://databricks.com/blog/2017/10/19/introducing-natural-language-processing-library-apache-spark.html",500))
Last refresh: Never
%md ## Spark NLP Latest ### 2019 Spark Summit [](https://www.youtube.com/embed/ziUrLnJ_Y8A) ### 2018 Data Science Summit [](https://www.youtube.com/embed/qG_bdbpOM-Y) #### 2018 Spark Summit - https://databricks.com/session/apache-spark-nlp-extending-spark-ml-to-deliver-fast-scalable-unified-natural-language-processing #### 2017 Spark Summit [](https://www.youtube.com/embed/k5X12mdEvb8) More videos, podcasts, articles, docs, etc. here: - https://nlp.johnsnowlabs.com/videos - https://nlp.johnsnowlabs.com/articles - https://nlp.johnsnowlabs.com/docs/en/quickstart
Spark NLP Latest
2019 Spark Summit
2018 Data Science Summit
2018 Spark Summit
2017 Spark Summit
More videos, podcasts, articles, docs, etc. here:
Last refresh: Never
displayHTML(frameIt("https://nlp.johnsnowlabs.com/articles", 500))
Last refresh: Never
%md ## Some resources to self-learn Scalable NLP - Spark NLP Walkthrough, powered by TensorFlow - https://medium.com/@saif1988/spark-nlp-walkthrough-powered-by-tensorflow-9965538663fd - Mueller Report for Nerds! Spark meets NLP with TensorFlow and BERT - https://hackernoon.com/mueller-report-for-nerds-spark-meets-nlp-with-tensorflow-and-bert-part-1-32490a8f8f12 - LDA Topic Modeling in Spark MLlib - https://medium.com/zero-gravity-labs/lda-topic-modeling-in-spark-mllib-febe84b9432
Some resources to self-learn Scalable NLP
- Spark NLP Walkthrough, powered by TensorFlow
- Mueller Report for Nerds! Spark meets NLP with TensorFlow and BERT
- LDA Topic Modeling in Spark MLlib
Last refresh: Never
%md # URLs that need to be descended further and scattered to other sds-2-x notebooks https://www.oreilly.com/ideas/comparing-production-grade-nlp-libraries-training-spark-nlp-and-spacy-pipelines https://blog.dominodatalab.com/comparing-the-functionality-of-open-source-natural-language-processing-libraries/ https://databricks.com/blog/2018/09/11/introducing-flint-a-time-series-library-for-apache-spark.html https://databricks.com/blog/2018/11/16/introducing-new-built-in-functions-and-higher-order-functions-for-complex-data-types-in-apache-spark.html # NLP https://databricks.com/blog/2019/05/20/new-videos-from-databricks-academy-introduction-to-natural-language-processing-latent-semantic-analysis.html https://www.analyticsvidhya.com/blog/2017/01/ultimate-guide-to-understand-implement-natural-language-processing-codes-in-python/ https://databricks.com/blog/2017/10/19/introducing-natural-language-processing-library-apache-spark.html https://nlp.johnsnowlabs.com/docs/en/quickstart https://github.com/JohnSnowLabs/spark-nlp-workshop/tree/master/databricks https://medium.com/zero-gravity-labs/lda-topic-modeling-in-spark-mllib-febe84b9432 https://medium.com/@saif1988/spark-nlp-walkthrough-powered-by-tensorflow-9965538663fd https://nlp.johnsnowlabs.com/docs/en/models https://hackernoon.com/mueller-report-for-nerds-spark-meets-nlp-with-tensorflow-and-bert-part-1-32490a8f8f12 - NLP: INTRO, word2vec, cosine similarity, ... https://github.com/marcopeix/Deep_Learning_AI/blob/master/5.Sequence%20Models/2.Natural%20Language%20Processing%20%26%20Word%20Embeddings/Remove%20Bias.ipynb Medium blog https://link.medium.com/XpqYfWbdqX - NLP: Hierarchical Attention nlp Word order and context https://www.cs.cmu.edu/~hovy/papers/16HLT-hierarchical-attention-networks.pdf Medium https://link.medium.com/FeVADtNeqX GitHub code https://github.com/jrpresta/website - DL latest: Google AI Efficient Nets Medium https://link.medium.com/bySTooLfqX Code https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet Blog https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html - Gaffer https://github.com/gchq/Gaffer/blob/master/example/road-traffic/README.md
URLs that need to be descended further and scattered to other sds-2-x notebooks
https://www.oreilly.com/ideas/comparing-production-grade-nlp-libraries-training-spark-nlp-and-spacy-pipelines https://blog.dominodatalab.com/comparing-the-functionality-of-open-source-natural-language-processing-libraries/
https://databricks.com/blog/2018/09/11/introducing-flint-a-time-series-library-for-apache-spark.html https://databricks.com/blog/2018/11/16/introducing-new-built-in-functions-and-higher-order-functions-for-complex-data-types-in-apache-spark.html
NLP
https://databricks.com/blog/2019/05/20/new-videos-from-databricks-academy-introduction-to-natural-language-processing-latent-semantic-analysis.html https://www.analyticsvidhya.com/blog/2017/01/ultimate-guide-to-understand-implement-natural-language-processing-codes-in-python/ https://databricks.com/blog/2017/10/19/introducing-natural-language-processing-library-apache-spark.html https://nlp.johnsnowlabs.com/docs/en/quickstart https://github.com/JohnSnowLabs/spark-nlp-workshop/tree/master/databricks
https://medium.com/zero-gravity-labs/lda-topic-modeling-in-spark-mllib-febe84b9432
https://medium.com/@saif1988/spark-nlp-walkthrough-powered-by-tensorflow-9965538663fd
https://nlp.johnsnowlabs.com/docs/en/models
NLP: INTRO, word2vec, cosine similarity, ... https://github.com/marcopeix/Deep_Learning_AI/blob/master/5.Sequence%20Models/2.Natural%20Language%20Processing%20%26%20Word%20Embeddings/Remove%20Bias.ipynb Medium blog https://link.medium.com/XpqYfWbdqX
NLP: Hierarchical Attention nlp Word order and context https://www.cs.cmu.edu/~hovy/papers/16HLT-hierarchical-attention-networks.pdf Medium https://link.medium.com/FeVADtNeqX GitHub code https://github.com/jrpresta/website
DL latest: Google AI Efficient Nets Medium https://link.medium.com/bySTooLfqX Code https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet Blog https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html
Gaffer https://github.com/gchq/Gaffer/blob/master/example/road-traffic/README.md
Last refresh: Never
SDS-2.x, Scalable Data Engineering Science
Last refresh: Never