Deep Learning for Language Analysis Summer School

By

I attended the Deep learning summer school for language analysis at Universität zu Köln from 9 to 13 September. The summer school was a collaboration between the Qualitative Modeling of Complex Systems group and the Department for Digital Humanities.

The venue of the conference was at the beautiful and green campus of University of Cologne.

Unlike last year when the summer school was titled Machine Learning for Language Analysis, this year the focus was on Deep learning (Neural Networks). Deep learning is gaining momentum due to the huge amount of data available and the rising computing capability of machines. Therefore, applying deep learning algorithms has now become prevalent in Natural language processing applications.

The summer school was comprised of two sections. In the first section we were introduced to deep learning and its use in text classification applications such as sentiment analysis, and text generation. The instructors Dennis Demmer and Johanna Binnewitt started with a theoretical introduction of neural networks for deep learning in language analysis applications. In the hands-on session, we used Keras and Tensorflow libraries in Python to implement a multi-layer perceptron model to identify sentiments in a Movie review database.

The second section was organized as three parallel sessions:

Session A: DEEP LEARNING WITH AUDIO & SPEECH DATA

Session B: DEEP LEARNING FOR NLP & ARGUMENT MINING

Session C: DEEP LEARNING IN TEXT RECOGNITION

I participated in the Text Processing and argumentation mining session due to its close connection to my research.

Dr Johannes Daxenberger from Technische Universität Darmstadt started this session with an introduction to word embeddings -a neural network based representation of words- (my colleague Ekaterina Kamlovskaya has written a blog post on word embeddings here). In the hands-on session we practiced how to train a word embedding with the Word2Vec algorithm implemented in the genism library in Python. We also experimented with pre-trained word-embeddings and how to find word similarities using them. Furthermore, the instructor gave an overview of different types of Neural Networks including Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) and their applications in Natural Language Processing (NLP). Convolutional Neural networks are mostly applied in the image processing domain, however today they are also being used in NLP applications because of their capability to capture some context structures by convolving filters on a sequence of words. In the practical session, we used the same movie review dataset from the introduction session to classify sentiments with a CNN model implemented using keras neural network library in Python.

The last session of Text Mining workshop was dedicated to argumentation mining. Argumentation mining is a research field in Text Analysis with the aim of detecting and analysing argument structures in textual resources. Its applications cover legal domain, persuasion technology, essay writing, political speeches and etc. The UKP group at Technische Universität Darmstadt published an online demo on web for searching controversial topics with the pro and con arguments for and against the aspects of topics.

On the last day of the summer school we had an introduction session to the NVIDIA pedagogical platform for deep learning. This platform provides various courses on Deep learning in different domains.

After the NVIDIA session, we had a meet and greet with several recruiting start-ups and companies based in Cologne.

I enjoyed participating in this summer school very much. If the summer school is going to be held next year in September, I strongly recommend it especially for last year Master students, or first year PHD students who want to follow a research in the field of text analysis.

Leave a Reply