The Faculty of Engineering at the North-West University (NWU) is rightly proud of the success of its new cohort of postgraduate students in the MuST (Multilingual Speech Technologies) research niche area.

All seven master’s degree students who joined MuST’s 2019/2020 deep learning programme had their degrees conferred on 30 June, with five of them having earned their degrees with distinction. They are Dewald Krynauw, Jacques Beukes, Christiaan Myburgh, Nuette Heyns and Coenraad Mouton.

The significance of these achievements lies in the fact that this postgraduate research programme is only three and a half years old. When MuST joined the Faculty of Engineering in January 2018 after the conclusion of the NWU's restructuring process, it had no postgraduate programme. The last set of MuST students had graduated in 2016, with PhD studies being located in a different faculty and in its previous programme, multilingual speech technologies for lesser resourced languages.

The new MuST research programme focuses on deep learning: these neural network-based architectures form an increasingly important set of tools in modern machine learning. The programme opened with only two students: Arnold Pretorius, who received his master’s degree in 2020 (also with distinction), and Tian Theunissen, who will receive his PhD at the 2021 winter graduation. In the second year of the programme, MuST took a leap of faith and accepted the entire group of shortlisted master’s degree applicants. Prof Marelie Davel, director of MuST, explains: “The seven final candidates impressed us – we did not know how to choose between them! Besides, we were by then convinced that we were onto something and we needed to expand our capacity. So we took them all in.”

The deep learning programme includes both theoretical and application-oriented studies: the recently graduated master’s degree students produced two studies exploring convolutional neural network theory (Coen Mouton and Christiaan Myburgh), two on the interpretability of space weather models (Dewald Krynauw and Jacques Beukes), two applications of speech processing (Rhyno Strydom and Nuette Heyns), and a bespoke solution for an industry partner (Dylan Lampbrecht). Studies also produced codebases and publications. All the students presented their work at conferences, with the work of three of the students having been selected for inclusion in special-edition journals.

Two of the graduating students felt no need for a break from research and are continuing with their PhD studies: Coen relocated to the MuST Hermanus lab and plans to continue focusing on deep learning theory. He aims to become a fully-fledged researcher in this field. Nuette has enrolled for a PhD at the NWU Unit for Languages and Literature in the SA context. She uses machine learning techniques to work on quantitative transtextuality detection, which is an umbrella term for different methods to reuse and reference text, for example, to associate current news and world events with social media posts.

The deep learning programme aims to prepare students for future careers that are expected to require much flexibility. As Dewald puts it: “The job I currently do did not even exist two years ago.” He is currently working as a blockchain developer with the Telos blockchain core team. He also develops DeFi (decentralised finance) apps.

Jacques has been bitten by the blockchain bug too and works with Dewald on DeFi apps. His immediate goal is to continue to sharpen his blockchain skills while looking for an opportunity to incorporate machine learning in his work. He hopes to continue to find opportunities to help create something interesting and useful.

Another master’s degree graduate to join the world of DeFi is Rhyno, who now works as a junior quantitative software developer at Invictus Capital. One of his roles includes scripting logic for algorithmic trading strategies in different markets. He aims to further his career in DeFi by using his newly acquired knowledge in data science to analyse financial markets. “I found my time with MuST very enriching. The research group emphasises clear thinking and is open to the sharing of ideas. They helped me gain a new perspective on things, and I definitely made some new friends along the way.”

Christiaan mainly works on developing single-page web applications and has taken up live streaming as a “passion project”. He reflects on his time as a master’s degree student: “MuST is filled with passionate people who never hesitate to help out and point you in the right direction. I am truly grateful to have had the opportunity to be part of such a caring and driven group.”

Dylan works as a machine-learning engineer/software developer on language models to understand and interpret large pieces of text. Although preferring diversity in his work, he foresees that he will continue to move in the direction of machine-learning research and development.

MuST’s first machine-learning alumnus, Arnold, has worked as a data scientist at Saigen, a speech analytics venture co-founded by a former MuST colleague, since the start of 2020.

***

Background

In 2018, Profs Marelie Davel and Etienne Barnard launched a new research programme focusing on deep learning. Internationally, deep neural networks (DNNs) have increasingly become the driving force behind breakthroughs in fields such as computer vision, natural language processing and bioinformatics. Deep learning models are particularly applicable when complex relationships must be inferred from large, high-dimensional data sets. In specific application fields, these systems are able to achieve and sometimes surpass human performance, making “narrow AI” more and more of a reality. These successes have inspired research into better algorithms, novel applications thereof, and a better understanding of DNNs.

The MuST deep learning programme was established with the vision of gaining an in-depth understanding of the inner workings of DNNs, contributing to some of the open theoretical questions in this field, and developing young researchers who can optimise the performance of these models for real-world applications with local significance.

Nuette, Dylan and Rhyno were supervised by Prof Etienne Barnard. Christiaan, Coen, Dewald and Jacques were supervised by Prof Marelie Davel, with Dewald and Jacques having been co-supervised by Dr Stefan Lotz from the South African National Space Agency (SANSA).

Prof Marelie Davel with most of the recent MuST graduates, at her inaugural lecture in 2019. From left are Rhyno Strydom, Jacques Beukes, Nuette Heyns, Cristiaan Myburgh, Prof Davel, Tian Teunissen, Arnold Pretorius, Dewald Krynauw and Coenraad Mouton. Insert: Dylan Lamprecht.