Overview / Usage

In a world where large sources of data are becoming more and more prevalent and time more and more precious, finding material that is clear and concise has become all the more important. This project allows students to focus on the new concepts that are being covered during lecture, without the additional distraction of taking notes. By quickly synthesizing the essence of a professor’s lecture, students are given a reliable study material and brief overview of the material covered in their course. Using notes and different encoding networks, this information can then be rephrased in many ways, giving the students as many options as possible until the information sticks.

Methodology / Approach

As the problem of converting text to speech is one that has already in essence been solved, it seems illogical to spend the time training our network to confront it. As such an accurate API will be assumed for tackling this first part of the challenge.
The main challenge of this project is accumulating the appropriate data. Finding dictations of the lectures of professors, and the corresponding notes to go with them, is no doubt quite difficult, so our model must be trained using additional resources, to get a representative data set. As plenty of notes exist for research papers, textbooks, and articles it seems only logical for our model to start its research there. By taking a large data set of student notes and organizing them by hand into datasets based on the note style and the type of resource/ specific book they are taken from, various neural networks can be trained. The networks would 1 produce certain note styles, 2 adjust the importance of certain information based on subject, and 3 ultimately synthesize various text sources into notes. By using word vectors combined with convolutional neural networks, a general model can be generated for all the data where certain nodes represent key ideas, and others represent more detailed information. To specialize the data, an additional layer will be added to this core neural network, which will be trained for the specific type of data the model is referring. By generalizing the results of this more specific network training, the model that translates the professor’s dialogue into notes can have a very close starting point to the actual optimal one, helping to solve the lack of data for this specific case.
Having a vast supply of student notes also opens an interesting opportunity where an encoder decoder network can be trained to essentially estimate what the professor said. As there is no doubt some ambiguity in the language that was used the meaning will be slightly changed in translation, but this may be a benefit. Oftentimes in order to understand a new concept you need a person to rephrase it, to put it into terms with which you are more familiar, and an implementation of an encoder decoder network in this area, could provide more colloquial and simplified explanations.

Technologies Used

Intel® Optimization for TensorFlow: improve performance
Intel Distribution for Python: quicken python libraries
Intel Neural Compute Stick 2: increase speed of network training