Genetic algorithms consist of several key components/features that work together to evolve a population of potential solutions to a problem. These components include:

1. Initialization: The process begins by creating an initial population of potential solutions (individuals) randomly or using some heuristic method.
2. Fitness Function: A fitness function quantifies how well each individual in the population solves the problem. It assigns a numerical score or fitness value to each individual based on its performance.
3. Selection: Selection involves choosing individuals from the current population to serve as parents for the next generation. Individuals with higher fitness values are more likely to be selected, as they represent better solutions.
4. Crossover (Recombination): Crossover is a genetic operator where pairs of parents are combined to create offspring. It mimics the process of genetic recombination in biology. The goal is to mix and match the genetic material of the parents to potentially create better solutions.
5. Mutation: Mutation is another genetic operator that introduces small random changes into the genetic information of individuals. It helps in maintaining diversity in the population and allows the algorithm to explore new regions of the solution space.
6. Termination Criteria: Genetic algorithms continue to evolve the population through multiple generations. Termination criteria, such as a maximum number of generations or reaching a satisfactory solution, determine when the algorithm should stop.
7. Replacement: After creating the offspring through crossover and mutation, the new generation replaces the old generation in the population. Replacement strategies can vary, but often, the least fit individuals are replaced.
8. Elitism: Some genetic algorithms incorporate elitism, which ensures that the best-performing individuals from the current generation are preserved in the next generation without undergoing crossover or mutation.
9. Parameters: Genetic algorithms involve several parameters, such as population size, crossover rate, mutation rate, and selection strategies. Tuning these parameters is crucial to the algorithm’s success and can impact its convergence and performance.

These components collectively enable genetic algorithms to iteratively explore and refine the population of solutions, with the ultimate goal of converging towards an optimal or near-optimal solution to the given problem. The process continues until a termination condition is met or a satisfactory solution is found.

As an example of use, this video presents an implementation of a particular genetic algorithm designed to efficiently address the Traveling Salesperson Problem (TSP). It illustrates how concepts from biology such as ‘survival of the fittest,’ ‘genetic diversity,’ and ‘mutation’ can be translated into code. The video concludes with a visual representation of the algorithm in action as it successfully tackles the TSP

This is just the tip of the iceberg and this lecture of Open MIT give more insights in the topic. In particular, it provides an exploration of genetic algorithms at a conceptual level. Three approaches to how a population evolves towards desirable traits are considered, culminating with assessments of both fitness and diversity. It also conclude with a brief discussion about how this space is abundant with solutions.

Genetic algorithms can be used in conjunction with a neural network in several ways to optimize and enhance the performance of neural network-based systems. Example includes the following:

1. Architecture Search: Genetic algorithms can help search for the optimal architecture of a neural network. They can evolve different network structures, including the number of layers, types of layers, and their connectivity, to find the configuration that best suits a given task.
2. Hyperparameter Tuning: Genetic algorithms can optimize hyperparameters such as learning rates, batch sizes, dropout rates, and weight initialization schemes for neural networks. This can improve the network’s training speed and overall performance.
3. Feature Selection: Genetic algorithms can be used to select the most relevant features or inputs for a neural network. By evolving subsets of input features, the algorithm can determine which features are most informative for a given problem.
4. Neuroevolution: In neuroevolution, genetic algorithms are used to evolve neural network weights and biases directly. Instead of traditional gradient-based training methods, genetic algorithms can evolve a population of neural networks and select the best-performing individuals based on their fitness.
5. Ensemble Learning: Genetic algorithms can create an ensemble of neural networks with diverse architectures or initializations. This ensemble approach often leads to improved generalization and robustness.
6. Transfer Learning: Genetic algorithms can optimize the transfer of knowledge from pre-trained neural networks to new tasks. They can evolve strategies for fine-tuning pre-trained models or selecting relevant layers for transfer.
7. Neural Network Optimization: Genetic algorithms can optimize the weights and biases of a neural network to fine-tune its performance for specific tasks, especially when traditional optimization techniques struggle with high-dimensional or non-convex parameter spaces.
8. Neural Architecture Search (NAS): Genetic algorithms can be used in NAS to automate the process of finding the best neural network architecture for a given task. NAS methods often involve evolving and selecting neural network architectures based on their performance on a validation set.

In summary, the integration of genetic algorithms with neural networks can help several problem-solving scenarios, as it offers a comprehensive solution that combines the global search capabilities of genetic algorithms with the learning and adaptation prowess of neural networks. In particular, genetic algorithms automate the optimization process, enabling efficient exploration of vast solution spaces and the discovery of optimal neural network architectures, hyperparameters, and weight configurations. This automation reduces the need for manual tuning and promotes diversity, ensuring that the best solutions are not prematurely discarded. Additionally, genetic algorithms facilitate ensemble learning, transfer learning, and fine-tuning, enhancing the adaptability, robustness, and performance of neural network-based solutions across a broad spectrum of domains and tasks.

The post On Genetic Algorithms as an Optimization Technique for Neural Networks appeared first on Francesco Lelli.

It was a really fun and formative experience where we had the possibility to interact with many colleagues across Europe. We can clearly declare the workshop successful as we had to cap the contribution to only 11 despite the very large contribution that we received (total 22, acceptance rage 50%). If that is not enough for declaring it a success, it also was the largest (most participated) event of the conference.

A big thanks also goes to our sponsors and keynote speaker that stimulated the conversation as well as provided a valuable industrial experience.

Here you can find a set of useful links in case you would like to profit of the outcomes:

• Call for paper for the first workshop on AI Beyond Efficiency
• Modafferi, S., Nuñez, M. J., Lelli, F., & Dalle Carbonare, D. (2022). Artificial Intelligence beyond Efficiency. CEUR Workshop Proceedings, 3214

Full report can be downloaded here: PDF Version of the report on the workshop

The paper that where discussed are available at the following links:

• Introducing Building Blocks for Industry 4.0, an Analytics Application for the Federated EFPF Platform
  Jose Maria Gonzalez Castro Robert Sanfeliu Prat, Francesc Bonada, Fernando Gigante Valencia
• AI Ethics for Industry 5.0 – From Principles to Practice
  Alexandru Constantin Ciobanu, Gabriela Meșniță
• A Practical Experience of AI Solution Used to Improve Varnishing Process Efficiency in Furniture Manufacturing
  Juan Del Agua, Gabriel Modia
• Industry5.0 and Sociotechnical Theory: Theoretical Underpinnings
  Nicholas Fair
• Relevance of Visualization and Interaction Technologies for Industry 5.0
  Ander Garcia, Marco Quartulli, Igor G. Olaizola, Iñigo Barandiaran
• Teaming.AI: Enabling Human-AI Teaming Intelligence in Manufacturing
  Thomas Hoch, Bernhard Heinzl, Gerald Czech, Maqbool Khan, Philipp Waibel, Stefan Bachhofner, Elmar Kiesling, Bernhard Moser
• On Exploring the Possibilities and the Limits of AI for an Interoperable and Empowering Industry 4.0
  Francesco Lelli
• Artificial Intelligence from Industry 5.0 perspective: Is the Technology Ready to Meet the Challenge?
  Igor García Olaizola Marco Quartulli, Ander Garcia, Iñigo Barandiaran
• Towards Zero-Defect Manufacturing: Machine Selection through Unsupervised Learning in the Printing Industry
  Sotirios T. Spantideas, Anastasios E. Giannopoulos, Nikolaos C. Kapsalis, Angelos Angelopoulos, Stamatis Voliotis, Panagiotis Trakadas
• Towards Industry 5.0 – A Trustworthy AI Framework for Digital Manufacturing with Humans in Control
  Usman Wajid, Alexandros Nizamis, Victor Anaya
• On Developing Human Centric Digital Twins in Industry 4.0 and Beyond
  Akpofure A. Enughwure, Francesco Lelli

As organizer I hope that the connections that the participants build will remain strong, will grow, and will materialize in a tangible advance of the field.

First Workshop on AI Beyond Efficiency: proceedings of the event. #ArtificialIntelligence #workshops #interoperability #Industry40 #BigData #MachineLearning
Share on X

The post AI Beyond Efficiency: note on the event appeared first on Francesco Lelli.

A knowledge graph is also known as a semantic network and represents a network of real-world entities. In other words represent “reality” as a combination of objects, events, situations, or concepts taking into account the relationship between them. This information is usually stored in a graph database and visualized as a graph structure, prompting the term knowledge “graph.”. The figure below can give you a visual idea of how these entities interlink among each other.

Intuitively, we can note that knowledge graphs take advantages of triples: subject-verb-qualifier in order to infer relationships among the data.

The knowledge graphs is at the core of the semantic web vision and standards like RDF and query languages like SPARQL are maintained by the w3c. A set of pointers for more information is available in this paper that I wrote specifically for interoperability in industry 4.0 and internet of things. However sections 2.3. (of the paper) give an overview of the semantic stack.

Existing tools and technologies that use knowledge graph

A core set of technologies that are currently in productions and used companies like Google, Facebook, Apple includes the following:

• Wikidata: Wikidata is a collaboratively edited multilingual knowledge graph hosted by the Wikimedia Foundation. It is a free and open knowledge base that can be read and edited by both humans and machines.
• schema.org is an organization that publishes documentations and guidelines to using structured data mark-up on web-pages. Its main objective is to standardize HTML tags to be used by webmasters for creating rich results about a certain topic of interest.
• Google Knowledge Graph is a knowledge base used by Google and its services to enhance its search engine’s results with information gathered from a variety of sources.

These technologies are strictly focusing on the data instead and the video below share a practical approach that focus on the data and how to combine them. Note that it does not consider any AI approach but focus on how to combine existing available data and their relationships.

However, it contains very nice examples and give an intuitive idea on how Wikidata, Schema.org and Knowledge graph works.

Research on Knowledge Graph and Artificial Intelligence

Now that we have an idea of what are the basic techniques for representing information we need to understand how we can use machine learning techniques for enriching these information and the expressiveness of the models. What follows is a set of seminars that discuss the knowledge graph from a more theoretical point of view and consider open research questions. This is the first part:

Below you can find the second part. It also contains very nice real life case studies. Including how Siri works!

If you are curious and would like to know more about the topic I encourage you in browsing the YouTube channel for more videos.

On #Knowledge Graph and #ArtificialIntelligence : how to combine data and models for better information. #BigData #DataScience
Share on X

The post On Knowledge Graph and Artificial Intelligence appeared first on Francesco Lelli.

Mentor at TiU: Francesco Lelli 

Mentors at Talpa: Anca Dumitrache and/or Ricardo Fabian Guevara

There are some internships opportunities available at Talpa. You will have the opportunity to develop your master’s thesis in collaboration with the AI division of one of the most innovative media companies in the Netherlands .

If you have a go-get attitude with the desire to expand your knowledge and expertise in the area of big data and artificial intelligence, this is probably the internship that you are looking for.

Topics for this thesis include Natural Language Processing (NLP), Machine Learning and Predictors. At the same time you are encouraged to propose your own idea as well.

Knowledge and Skills:

Programming, preferably Python, and/or statistical skills.

Interested in Joining Talpa?

Here you can find a short presentation of the company:

Where to Read More:

The following ideas are possible internship opportunities and related topic for your thesis. You may want to use them for forming an idea of the kind of jobs that you will be doing at Talpa as well as for developing your research proposal (here you can find a few tips for that).

Swimlane for Trending on Social Media

Both KIJK (video streaming platform) and JUKE (audio streaming platform) present their content separated into swimlanes on the front page. For instance, one swimlane contains a list of TV shows that were popular in the previous days. We would like to create a new swimlane that contains items that were trending on social media. Taking Twitter as a data source, the project will go through the following steps:

• (1) extract tweets about popular TV shows and/or radio shows,
• (2) perform entity linking to match them to the shows in our database,
• (3) aggregate the results to get the most popular shows in one swimlane.

Project: KIJK and/or JUKE

Automatic Teaser Tweet Creation

Starting with a (textual) description of a TV show episode or radio program, we would like to generate teaser tweets about the show that are meant to generate anticipation on social media. The underlying task would be a summarization problem, where the program description is mapped to a short tweet about it. The tweet should contain relevant information about the program, but not reveal any spoilers. Either an extractive (entity + relation extraction) or abstractive method could be applied.

Source: https://www.aclweb.org/anthology/N19-1398/

Project: KIJK and/or JUKE

Exploring Pair-wise Learning-to-Rank

Talpa current recommender system uses ALS, a point-wise learning-to-rank approach, where the learning objective is based on modeling the score of a given item (i.e. similarly to how regression works). Alternative methods of doing learning-to-rank are pair-wise (learning objective is to model the ranking of a pair of items relative to each other) and list-wise learning to rank (learning objective is calculated over the entire list of items). The project goal is to investigate different learning objectives and find out:

• (1) how they perform relatively to the point-wise method,
• (2) if there are subsets of data where this method works better/worse.

Source:https://medium.com/@nikhilbd/pointwise-vs-pairwise-vs-listwise-learning-to-rank-80a8fe8fadfd

Project: KIJK and/or JUKE

Automatic Playlist Generation

JUKE music player features a lot of non-stop music playlists that are manually created by an editor, usually selecting music from a given genre (e.g. hard rock non-stop radio). We would like to see whether these playlists can be generated by AI and what the quality is. This can be approached as a song clustering problem, where the feature space could contain the genre, artist, as well as other audio features.

Project: JUKE

Google AdWords for Video

Advertisers can buy Google search keywords to show their ad in conjunction with them. We would like to see whether it is possible to do something similar within videos, too. In this way, contextual advertising in different medias becomes feasible. The steps involved in this project are:

• (1) generate textual metadata of video (either based on transcript, or other features of the video; there might be some existing video metadata as well),
• (2) match video textual metadata with ad keywords.

Exploring Seasonal Trends as RecSys Features

Our domain experts know that both video and audio streaming trends are highly influenced by seasonality (e.g. Sky Radio becomes very popular over Christmas). The goal of this project is to:

• (1) identify these trends by studying user listening data, then
• (2) incorporating these trends into our prediction model (e.g. recommend Sky Radio to fans of Christmas music, but only over the Christmas holidays).

Challenge: a lot of data may be needed to accomplish this.

Project: KIJK and/or JUKE

Radio Station Embeddings

Goal: produce embedding with various sizes similar to Glove but for radio stations that reflect radio station similarity with euclidean distance.

Value: many of Talpa challenges, like radio to radio similarity, could be tackled with this approach. Our RecSys can also use the radio embedding as a feature to determine if a radio channel is a good match for a user. This could also greatly alleviate the cold start problem when we introduce new radio stations.

Approach: this is an unexplored problem but there has been previous research and practical work done on audio embeddings. In the simplest form, a radio station embedding could be just a bag of features like genre frequencies, languages and origins of artists, but also properties from the target audience like age range. Going beyond that, it could use an average of the embeddings of representative songs on it in a given past period. Another valuable asset of Talpa is its understanding of usage behavior, so radio station features could also be calculated based on the type of person who listens to it, the frequency of listening and other seasonal behavior.

At the same time, there is a temporal aspect to take into consideration when analyzing user preference of radio stations. Therefore, we cannot expect the embeddings to remain the same forever. Consequently a refresh period is to be expected (even word embeddings need to account for word semantic drifting, only their refresh window is larger).

Project: Juke

Podcast Embeddings for JUKE

Similar to the radio embeddings project, Talpa is interested in creating embeddings for the podcasts available on JUKE. Podcast recommendations suffer from cold start problem even more than radio, since there is usually a high volume of items that are published continuously. The features that can be used for podcast embeddings are also slightly different than typical audio embedding features based on e.g. musical genre.

Project: Juke

Interested in a Talpa internship? Check also the Following:

Webinar about Talpa Internship Opportunities (Available for Tilburg Student Only)

Interested in Applying?

Send an email to Francesco Lelli with CV, Short motivation letter, project(s) that interest you and (optional) draft research proposal. In the case you would like to propose a particular project please put extra attention to your research proposal.

A selected list of candidates will be interviewed by TALPA that has the ultimate saying on accepting you for an internship. Succes, Good luck, In Bocca al Lupo!

The post Master Thesis Plus Talpa Internship Opportunities in Big Data and Artificial Intelligence appeared first on Francesco Lelli.