Many of us began our open source journey by fixing a typo, fixing a bug, improving documentation, or adding a feature. That one merged pull request marked the start of our contributions to open source.

But what happens after you become a regular contributor to that project? If you are someone who strives to grow in every situation, you’ll soon realize that being a regular contributor is just the beginning and often not enough. There is much more to open source; it is an entire world of learning, ownership, and leadership.

Disclaimer: If you’re one of those who are tired of just being a regular contributor and looking to go beyond just contributions, then this article is for you. It’s not a guide, but it highlights what lies beyond contributions, how to embrace maintainership, learning in public, and personal growth.

Curiosity as a Compass

Curiosity is often an underrated attribute to have, and yet it can be a catalyst for growth. It’s easy to fall into the habit of waiting for specific tasks to be assigned to you or for “good first issues” that have a clear scope and are easy to tackle.

True growth begins when you cultivate a mindset that seeks to understand the “why” behind your contributions. Consider asking yourself questions that challenge your knowledge, such as:

• What problem is this pull request (PR) trying to solve, and what challenges does this project aim to address?
• How do maintainers determine which features are needed for this project?
• How can I improve the user or onboarding experience?
• What are the current best practices for maintenance and project management?
• Where are the unnoticed gaps that need attention?

Each question sparks a new opportunity. Allowing yourself to be curious about how things are and how they can be improved can inspire you to take the initiative beyond your assigned tasks and begin making broader, innovative contributions.

Learning in Public

While contributions and mentorship programs are important aspects of open source, they are not the only paths to involvement. Often, after completing a program, participants may feel stuck, unsure of what to do next, and might end up moving on to another program. However, many successful maintainers did not rely solely on structured programs for their growth; they thrived by learning in public.

Here are some pathways I took to elevate my presence in the project and further my growth and learning. These are not the only options available, but they are practices I still use in my learning journey that you might consider exploring:

1. Participate in Community Meetings and Discussions: Many projects have various communication forums. Utilize these spaces to increase your visibility. Engage in discussions, share your insights, and learn from others. Such involvement can offer unique perspectives and foster relationships that create additional learning opportunities.
2. Volunteer: Volunteering without expectations often leads to unexpected results. Even small contributions can provide valuable experiences you wouldn’t have gained otherwise. For example, my journey in community building began after I hosted a few sessions at an AsyncAPI Online Conference. This experience was new for me and encouraged me to get more involved, helping me establish my presence not just through writing documentation but also through active participation in other community activities.
3. Drive Project Initiatives: Similar to volunteering, identify areas where the project could improve or expand. Look for opportunities to take the lead, such as becoming a release coordinator or champion. Taking on such initiatives positions you as an essential team member and helps cultivate your leadership skills.
4. Learn in Public: Sharing your learning journey with others not only fosters your growth but also inspires and educates those around you. Being transparent about your challenges and discoveries can create a ripple effect, attracting more contributors and solidifying your role as a knowledgeable resource in the community. You can do this through writing blog posts, creating tutorials, or giving talks at meetups or conferences.
5. Seek, Don’t Just Wait for Mentorship: While mentorship programs can provide valuable guidance, don’t rely solely on formal arrangements. Approach community members whose work you admire and ask for advice. Forming connections based on shared interests can lead to organic mentorship and foster collaborations that accelerate your growth.

By taking these steps, you position yourself as a resource, not just a participant. You’ll be surprised at how much faster you grow when your learning is visible.

The Leap to Maintainership

The greatest shift starts when you decide to move beyond being a regular contributor and aim to become a maintainer. That’s when curiosity meets responsibility. Instead of waiting to be assigned tasks, you start creating issues, reviewing your peers’ work, and learning how to properly triage tasks within a project.

You get to contribute to shaping the direction of the project, onboard other new contributors, collaborate, and actually gain experience on how to work in a distributed environment, and also foster innovation.

Making such a decision and leap will also challenge your thinking and help you view things from a different perspective. It cultivates a mindset focused on sustainability and long-term success.

Ultimately, growth in open source is not linear; it is a cycle of curiosity, experimentation, ownership, and responsibility. Every interaction and effort counts. Your unique perspective is valuable, and a whole world awaits you to explore beyond regular contributions.

Fuel your passion with curiosity, and let it take you further than you ever imagined. Get out there, explore, and let your journey inspire others, too!

Originally published at https://thulieblack.github.io on September 25, 2025.

In this current remote work setup, employers have upgraded their standards when it comes to hiring programmers. They look for individuals who possess the ability to use multiple functional paradigms to solve business problems. Functional programming has gained popularity due to its adaptability and efficiency to solve real-world problems. This article will cover the core concepts and the advantages of functional programming.

What is functional programming?

Functional programming is a declarative programming paradigm style where one applies pure functions in sequence to solve complex problems. Functions take an input value and produce an output value without being affected by the program. Functional programming mainly focuses on what to solve and uses expressions instead of statements. Functional programming excels mostly at mathematical functions where the values don’t have any correlation and doesn’t make use of concepts like shared state and mutable data used in object-oriented programming.

Functional Programming Concepts

Functional programming is built with various core concepts which we will explore below:

● First-class functions

First-class functions in functional programming are treated as data type variables and can be used like any other variables. These first-class variables can be passed to functions as parameters, or stored in data structures.

● Recursion

Unlike object-oriented programming, functional programming doesn’t make use of “while” or “for” loops or “if-else” statements. Functional programs avoid constructions that create different outputs on every execution. Instead, recursive functions call themselves repeatedly until they reach the desired state or solution known as the base case.

● Immutability

In functional programming, we can’t modify a variable after being created. The reason for this is that we would want to maintain the program’s state throughout the runtime of the program. It is best practice to program each function to produce the same result irrespective of the program’s state. This means that when we create a variable and assign a value, we can run the program with ease fully knowing that the value of the variables will remain constant and can never change.

● Pure functions

Pure functions form the foundation of functional programming and have two major properties:

• They produce the same output if the given input is the same
• They have no side effects Pure functions work well with immutable values as they describe how inputs relate to outputs in declarative programs. Because pure functions are independent this means that they are reusable, easy to organize, and debug, making programs flexible and adaptable to changes. Another advantage of using pure functions is memoization. This is when we cache and reuse the results after computing the outputs from the given inputs.

● High-order functions

A function that accepts other functions as parameters or returns functions as outputs is called a high-order function. This process applies a function to its parameters at each iteration while returning a new function that accepts the next parameter.

Advantages of functional programming

● Easy to debug

Since pure functions produce the same output as the given input, this means they aren’t any changes or any other hidden output produced. Functional programming functions are immutable, this also means that it’s easier to check for errors in code faster.

● Lazy evaluation

Functional programming adopts the lazy evaluation concept, whereby the computations are only evaluated the moment they are needed. This gives programs the ability to reuse results produced from previous computations.

● Supports parallel programming

Because functional programming uses immutable variables, creating parallel programs is easy as they reduce the amount of change within the program. Each function only has to deal with an input value and have the guarantee that the program state will remain constant.

● Easy to read

Functions in functional programming are easy to read and understand. Since functions are treated as values, immutable, and can be passed as parameters, it is easier to understand the codebase and purpose.

● Efficient

Since functional programs don’t rely on any external sources or variables to function, they are easily reusable across the program. This makes them more efficient as there isn’t extra computation needed to source the programs or run operations on runtime.

Drawbacks of Functional Programming

● Terminology Problems

Because of its mathematical roots, functional programming has a lot of terminologies that may be difficult to explain to a layman. Terms like “pure functions” can easily scare off people looking to learn more about functional programming.

● Recursion

Although recursion is one of the best features in functional programming, it is very expensive to use. Writing recursive functions requires higher memory usage which can be costly.

Functional programming languages

Now as you can imagine not all programming languages support functional programming. Some languages however were designed to be specifically for functional programming, while others do support both functional and object-oriented programming. Below is a list of some of these programming languages:

● Haskell — Made specifically for functional programming, Haskell is a statically typed programming language. It compiles code faster, is memory safe, efficient, and easier to read.

● Python — Although python supports functional programming, however, it was designed to prioritize object-oriented programming first.

● Erlang — Although it is not popularly used like Haskell, Erlang is best suited for concurrent systems. Messaging apps like WhatsApp and Discord make use of Erlang because of its scalability.

● JavaScript — Similar to Python, JavaScript isn’t specifically designed for functional programming. However, functional programming features like lambda expressions and attributes are supported making JavaScript a top-used language among multi-paradigm languages.

● Clojure — Clojure is a functional programming language that provides tools to avoid mutable states. Although it supports both mutable and immutable data types, it is less strict than other languages.

● Scala — Supporting both functional and object-oriented programming, Scala was designed to address the shortcomings of Java. It also comes with a built-in static-typed system similar to Haskell.

Conclusion

As we have covered some general core concepts found in functional programming, there is so much more to be explored. Learning more about functional paradigms can give you the leverage to use their tools and techniques to solve business problems.

I hope you found some value while reading this article and if you have any questions or suggestions please drop them in the comment section. See you in the next article, till then take care.✌🏿✌🏿

Originally published at https://thulieblack.hashnode.dev on January 24, 2023.

Introduction

Mankind is constantly seeking ways to make life better. Many have taken different paths in search of knowledge that transforms the way we do things. Such quests have led to the rise of the fourth industrial revolution. Because of this, we have seen lives improve, and industries worldwide are adopting this into their businesses. With this innovative and cutting-edge technology, the economy across the globe has also improved exponentially. This article will focus on how the fourth industrial revolution has impacted livelihood in this 21st century.

What is the fourth industrial revolution?

The fourth industrial revolution is the integration of technologies that are smart, fast, scalable, and more reliable to execute different tasks. Many companies have embraced this new revolution and the inventions that come with it.

Benefits of the fourth industrial revolution

In each and every phase of evolution, benefits are drawn from it. The fourth industrial revolution has come with a lot of advantages. These advantages are as follows:

• Increased productivity — with many innovations like programmable machines, production is done faster.
• Good censored decisions — with techniques like data tools, companies can make decisions wisely and accurately.
• Quality products — they aim for continuous improvement due to the latest technologies being developed. Quality products are created using cutting-edge technologies, ultimately improving the lifestyle of the consumers
• Safety — the fourth industrial revolution has reduced risks in operations when the environment is considered hazardous. With machines, workers can work independently without worrying about their safety.

Fourth industrial technologies

We cannot turn a blind eye to the most important thing that led to the rise of the fourth industrial revolution. Innovation has been the center of it, and we, therefore, need to give credit to some of the technologies that play a massive role in this period. These technologies are mentioned below;

1. Internet of Things (IoT)

Information is considered the most crucial part of anyone seeking to understand something. Internet of Things has been the hub of the development of devices connected to the internet. The number of devices being developed under this technology is increasing exponentially. Information is widely available and is changing how people perceive and comprehend things. In general, the Internet of Things has revolutionized various fields of work.

2. Artificial intelligence (AI)

Artificial intelligence is the most effective technology ever to be invented. Applications such as speech recognition, Google assistance, and Socratic have impacted many people’s lives. Artificial intelligence inventions have also helped improve the productivity of many remote workers and support work-life balance.

3. Virtual Reality and Augmented Reality

Virtual reality and augmented reality are technologies that are widely associated with gaming. However, many have used them for various purposes that benefit society. For example, one can easily locate a place and know its exact features without needing to be physically there. This saves both time and money. Additionally, these technologies have gone as far as training student pilots to fly a plane before finally getting into a real one.

4. Blockchain

In this age of technology, protecting your information is essential. Blockchain technology ensures that your information is safe and secure. Many industries have been drawn to the technology’s unique features, such as digital ledger and cryptography. Blockchain is still new and has potential for growth, not limited to cryptocurrencies. In a few years, industries such as insurance, banking, and healthcare, among others, will change the way of securing data with the help of blockchain.

5. Robotics

The creation of machines has been the focus of humankind since the first revolution. Robotics is another advanced step that creates intelligent machines that assist humans in various industries. Robotics includes automation and mass production and can be achieved with minimal labor. Although the development of robots is still a debate regarding human job security, robotics creates various opportunities under the fourth industrial revolution.

Conclusion

The fourth industrial revolution marks the beginning of even more remarkable achievements in human history. Everything, including humans, is getting smarter by the day. However, we must keep in check that every invention in this era benefits and improves mankind’s welfare.

Introduction

Inventions have been a major contributor to the world we see now more than they were a decade ago. From Albert Einstein, Isaac Newton to Mark Zuckerberg, these are a few names of individuals who trailblazed some inventions that have shaped over the years into what we see today. However, there are many incredible inventions made by women of which many of us are unaware of. In this article, we are going to outline groundbreaking inventions made by women in tech who enabled their desires to come to reality.

1. Hedy Lamar — WiFi

Wifi is a powerful tool in this modern age that has enabled impossible situations possible. This invention has led people to connect globally without the limitations of distance. Virtual meetings, working remotely, and bringing the world into the comfort of your own home are now possible due to Wifi. Most of us are now enjoying it, however, it is also important to appreciate and understand the efforts of individuals that made this luxury possible.

Hedy Lamar, an actress by profession in the 1920s, brought about this idea that led to what we call Wifi technology today. This came from her desire to change the outcome of the war. with her co-Inventor George Antheil, Lamar helped the navy to be able to communicate through different frequencies without leaking information to their enemies.

The frequency hopping technology was later patented in the 1940s giving rise to the first secure military encryption technology. This technology evolved to what we now benefit from as Wifi. Not only did Hedy exhibit her beauty, but she proved to be more of beauty with brains.

2. Shirley Ann Jackson — Caller ID

Shirley Ann Jackson was an American Physicist who invented the caller ID technology. She worked in a telecommunications company in 1976 where she invented the called ID. Due to her extensive knowledge coupled with her research abilities, she was able to devise such a helpful technology.

Every person can concur that the caller id has been very useful in our daily lives. We can now save numbers with the names of people on our devices and recognize them while calling. This has been seen to be a useful mechanism over the years and has saved a lot of time instead of spending it memorizing numbers. With confidence, we can conclude calling people is much easier and simple, thanks to Shirley Ann Jackson.

3. Ann Tsukamoto — Stem Cell Isolation

While conducting medical research, it is crucial to eliminate all the unwanted factors and concentrate on the subject to be tested. Clinical properties within the human body tend to interfere when elimination is not done. This results in vague conclusions and makes it difficult to advance that project.

Ann Tsukamoto developed an interest in this matter when she worked in a company specializing in stem cell research and development. Tsukamoto invented a cutting-edge technology of isolating stem cells in order to conduct clinical experiments. With her technology, she discovered a hematopoietic cell in the blood, which led to some of the stem transplant technologies focusing on cell-related diseases.

4. Roberta William — Graphic Computer Game Designer

After a long day’s work, we need time to relax to gather the energy for the following day. There are different ways people relax, some do yoga, and some soak themselves in a hot bath. Some find gaming as something to relax too.

Roberta Willam, known as the gaming genius was able to design and create some of the best-ever known adventure games in history. With her detailed storylines and complicated puzzles, Roberta was able to create memorable gaming experiences for people around the globe.

5. Radia Perlman — Spanning Tree Protocol

Radia Perlman played a role in what we call the Internet today. As a network engineer, her spanning tree protocol invention widely contributed to the development of the production of networks using ethernet. She ultimately contributed to the way data moves and is organized.

6. Marie Van Brittan Brown — CCTV

Imagine a patient goes missing in the hospital, and the hospital personnel cannot find the traces of the patient’s disappearance. This can be very traumatizing to the relatives of the patient and easily jeopardize the hospital’s reputation.

Thanks to Marie, a nurse who overcame this challenge by inventing CCTV. With it, we are now better positioned to keep our environment alert. Her works were unique; almost every company uses this technology for better security and accountability.

7. Ada Lovelace — Computer Algorithms

Born from a noble family working with the inventor Charles Babbage, Ada was able to translate what was Charle’s first general-purpose computer. She worked closely to interpret how machines worked. Babbage’s machine produced algebraic patterns and because of Ada’s fascination with mathematicians, she created the very first algorithm.

Conclusion

All these women in tech contributed their part to what we call modern-day technology. As we progress in technology, we look forward to the future, hoping for more extraordinary inventions. These women in their diversity were not limited to their careers and status. They strived to better the world despite all odds. Their inventions were a result of pure hard work, consistency, and resilience. The tech ecosystem can only get better moving forward.

Weights and Biases

Weight and bias are learning parameters that transform input data within the networks hidden layers. Both of these parameters are adjusted during training. Weight affects the amount of influence a change in the input will have upon the output while Bias mainly represents how far off the predictions are from the actual values.

source DeepAi

Learning Process

Neural networks are composed of layers of neurons (computational units) connected to each other. Each neuron transforms input data by multiplying the initial values by weights. The neurons have an activation function that defines the neurons output. The activation function is used to introduce non-linearity in the model and ensures that values passed in the network lie within the expected range. This process is repeated until the final output layer can provide predictions related to the task. Neural networks build knowledge from datasets where the right answer is provided in advance, then learn by tuning themselves to find the answers on their own in an iterating process of going ‘back and forth’ increasing the accuracy of predictions. The process of going ‘forth’ is called forward propagation, while the going ‘back’ is called back propagation.

Activation Function

Activation function is an algorithm used in neural networks which defines how the weighted sum of the input values is transformed into outputs. Activation functions are useful as they add non-linearity to the network. Non-linearity means that the output cannot be reproduced from a linear combination of inputs. Non-Linear functions are useful as they help networks learn complex information in order to provide accurate predictions/outputs. The reason we use non linear activation functions is because if we were to use a linear activation function then the neural network will just produce a linear function of inputs and no matter how many layers our neural network has, it will behave just like a single layer.

Let’s look at some of the commonly used non-linear activation functions.

Sigmoid

Forward Propagation

Forward propagation is the process where the input data is passed into the network where by all neurons apply their transformation(activation function) to the data they received from the previous layer of neurons and pass it to the next layer. The data flows in a forward direction till the final layer is reached and produces the output data(predictions). The input data is fed only in a forward direction and the data does not flow in reverse direction during the generation of the output data. These configurations are known as feed-forward networks which help in forward propagation. The reason why the data needs to only flow forward is because the reverse direction will cause a formation of a cycle thereby prohibiting the generating of output data(that’s why they are called feed forward networks).

Loss Functions

The next phase is to use the loss function to estimate the errors and measure the models predictions against the actual values. A loss function is a method used to optimize models with the intent of reducing the loss between predicted values and actual values. Ideally we want as less loss as possible, that is why it is important to adjust the parameters gradually till we achieve the desired results.

Back Propagation

Once the loss has been calculated the information is then propagated backward hence getting the name back propagation. It involves the calculation of the gradient going back ward through the feed forward network from the last layer to the first. This allows us to calculate the gradient of the loss function with respect to the weights of the model. Weights are updated individually to gradually reduce the loss function over training iterations. see image below

source toress.ai

What this process above represents is that we are making the loss as low as possible when we go back to iterating to the network. This technique is called gradient descent.

Gradient Descent

Gradient descent is an optimization algorithm used to train models by minimizing the loss function whenever the parameters are being updated. Until the loss function is close to zero or equal to, the model continues to adjust in order for the parameters to obtain as minimum as possible. In order for the gradient to reach a minimum, the learning rate should be set at an appropriate value. If the learning rate is too high, the minimum point will not be reached but if we set the learning rate too low the gradient will take time to reach the minimum point, that is why the learning rate shouldn’t be too high or too low.

Let’s look at the type of gradient descents

• Batch Gradient Descent
• Batch gradient descent calculates the error of each example within the training dataset but only updates the parameters after all the training examples have been evaluated. The advantage of batch gradient descent is that it is computationally efficient, produces stable error gradient and a stable convergence, but sometimes that convergence point isn’t the best the model can achieve.
• Stochastic Gradient Descent
• Stochastic gradient descent (SGD) runs a training epoch for each example within the dataset and it updates each training example’s parameters one at a time. The advantage of SDG is that it’s frequent updates allows us to have a pretty detailed rate of improvement but because of the frequent updates it becomes more computationally expensive than batch gradient descent.
• Mini-batch Gradient Descent
• Mini-batch gradient descent combines both batch gradient descent and stochastic gradient descent concepts. It splits the training dataset into small batch sizes and performs updates on each of those batches. This approach strikes a balance between the computational efficiency of batch gradient descent and the speed of stochastic gradient descent. This is the go-to algorithm when training a neural network and it is the most common type of gradient descent within deep learning.

Overfitting

Overfitting is a concept in machine learning where the model fits exactly against the training data and doesn’t perform well when it comes to unseen data. Generally we want models to perform well for both training and testing data(unseen data). Overfitting happens when the model trains for too long and starts to memorize the data. Low error rate and high accuracy in a model are good indicators of model over fitting. see example below

example of an overfitting model source Wikipedia

There are a few methods one can try to avoid overfitting :

1. Less training — reducing the training epochs or training time helps with overfitting by reducing the model noise and preventing the model in memorizing the data.
2. Reducing Neural Layers — A complex model is likely to overfitting therefore by reducing the neural network layers, the model becomes less complex and eventually reducing overfitting.
3. Dropout — Dropout is a form of regularization, a technique used to constrain models from learning complex data. Applying dropout helps by reducing or randomly ignoring neurons in a given layer during training hence reducing overfitting.

Conclusion

Finally deep learning plays a crucial role in our modern civilization. From demographic statistics to healthcare, deep learning has been a tool that enabled devising of viable solutions to address real life problems. It is very important for those that want to delve into this pool of unlimited knowledge to have a full comprehension of it, starting from the basics provided. With these few basic insights, l believe one is able to navigate a way to advance and create a brighter future that we all hope for with a magic touch of AI.

If you missed the previous sections of this article you can find

Resources

Originally published at https://thulieblack.hashnode.dev on November 30, 2021.

One hot encoding

One hot encoding is a method where data is encoded in form of vectors consisting of 0 beside 1 element. Each word encoded as a one-hot vector is unique meaning that each word in a sentence has a unique assigned vector. There aren’t any instances where two words share the same vector. see image below

In the previous article we touched on why we need to use floating point numbers as the preferred data type for tensors. We are now going to see this in practice. The image above shows a representation of a one-dimensional tensor but in order for us to feed this information to the model, we need to convert the vectors to a tensor(floating point).

But why go through the trouble of converting data types instead of just feeding the vectors to the model?

When we take a look at the dimension of the vector, we see that all the 0 elements don’t contain any information. This is a perfect example of memory wasted, remember each values has a storage property assigned to it. There isn’t enough information to describe the dimension because the goal is that the more information the better. This is why we need to convert the vector to a tensor in order to save memory storage and be able to pass meaningful data to the model.

Embeddings

We have established a sense of how one hot encoding works however the technique is only effective when it comes to small datasets. How then do we apply it to large datasets without running out of memory while providing more dimensionality? To solve this issue, we can simply implement word embedding. Word embedding is a dense vector of tensors that represents words in a multidimensional space, where words that have a similar meaning have similar representations. Let’s take for instance we have two sentences’i love my dog’ and ‘i love my cat’ , cat and dog will share a similar representation(encoding). see example below.

Let’s explore at two mainly used algorithms use in Word Embeddings

1. Embedding Layer
2. Embedding layer is a word embedding that has been learned from a neural network model. It can be understood as a look-up table that maps one hot vector’s indices to dense vectors(embeddings). The dimensionality of the embedding layer can be defined according to the task, and the weights are randomly initialized but gradually adjusted during training. Once trained, the learned word embeddings will roughly encode similarities between words.
3. Word2vec
4. Word2vec is a method that processes text to vectors of words. The word2vec is mainly based on the idea of representing words in their context. There are two major learning models that can be used as part of the word2vec in word embedding :

• Continuous Bag Of Words — this model learns the embedding by predicting the current word, based on its context. The intuition behind this model is quite simple, given a sentence like ‘I like my dog’, we chose our target word to be ‘like’ and our context words will be ‘i my dog’. What this model will do is take the distributed representations of the context words to try and predict the target word.
• Continuous Skip Gram — this model learns by predicting the surrounding words given in a current word. It takes the current word as input and tries to accurately predict the words before and after this current word. This model essentially tries to learn and predict the context words around the specified input word, given a sentence ‘let’s go play outside’, we can choose our input word to be ‘go’ and the model will try to predict ‘ let’s play outside’.

Both models are focused on learning about words given in their usage context where the context is defined by the neighboring words. The main benefit of these algorithms is that they take huge amounts of words in a large dataset, and produce high-quality embeddings that can be learned efficiently while using low storage space, providing more dimensions, and taking less computation time.

Tokenizers

Tokenizers is a technique that splits text, breaking it down into smaller pieces called tokens. For example, ‘The cat is sleeping’ can be split into four tokens ‘The’ ‘cat’ ‘is ‘sleeping’. see example below.

Not only do tokens function in the splitting and breaking down of text, but they also give machines the ability to read text. This plays a pivotal role when it comes to model building and text pre-processing.

text representation of tokenization

Let’s look at the three most commonly used tokenization algorithms and see how they work.

Spacy

Spacy is a natural language processing library used in tokenizing text in tensors. The input to the tokenizer is text and the output is a sequence of objects stored in a DOC object. The DOC object is a container that holds an array of tensors and for this to be constructed, a vocabulary(vocab) that contains a sequence of word strings must be created. Vocab is a storage class used to store data during tokenization in order to save memory. During tokenization, spacy tokenizes text by applying specific language rules. For example, punctuation that is found at the end of a word is usually split but in the case of a word like ‘N.Y.’ it remains as a single token.

Subword

As the name entails, subword tokenizes text into subwords. For example, ‘unfriendly’ can be split as ‘un-friend-ly’.One of the common methods subword uses is Byte Pair Encoding (BPE), which is popular when building transformer-based models. Subword uses character tokenization by taking one text data character at a time without grouping them. BPE helps subword by effectively tackling the concerns of running out of vocabulary words, and also strikes a good balance between performance and tokenization quality.

What is BPE

BPE is a data compression algorithm whereby the most frequent pair of bytes (word/token) of data is replaced by a byte that does not appear in the data. Let’s take for instance we have data aaabdaaabac which needs to be encoded. The byte pair aa occurs frequently, so we replace it with Z as it does not have any occurrences in our data. So now we will have ZabdZabac where Z = aa. The next common byte pair is ab so we’ll replace it with Y. We now have ZYdZYac where Z = aa and Y = ab. The only byte left is ac which occurs only once so we don’t encode it. We can use recursive byte pair encoding to encode ZY as X. Our data has now transformed into XdXac where X = ZY, Y = ab, and Z = aa. There’s no need to further compress the data as there are no more byte pairs appearing more than once. To decompress the data we start by performing replacements in reverse order. see example below

source jaketae

We have seen how BPE works in general but when it comes to text tokenization, things work differently as we don’t need any compressing. BPE ensures that the most frequent words are represented in the vocabulary as single tokens while less frequent words are broken down into two or more tokens. Suppose we have a preprocessed words , these words are spilt into characters. The bpe algorithm will look for the most frequent byte pairing, merge them and perform the same iteration until the iteration has reached the limit. That is why when it comes to machine translation, bpe is the most recommended tokenizer to do the job.

Sentence Piece

Sentence piece is a subword-independent text tokenizer and detokenize designed for neural-based text processing, where the size of the vocabulary is predefined before feeding to the neural model. lt implements subword segmentation algorithm’s as a parameter for the trainer. In its mission to achieve the process, sentence piece uses main components which are a normalizer, trainer, encoder, and decoder.

Normalizer is when the logical-equivalent characters of information/text/data is converted into simpler forms. From the normalized corpus, the subword segmentation model is then trained (Trainer). Encoder internally executes Normalizer to convert input tokenizers into a subword sequence with the subword model trained by Trainer. The Decoder corresponds to postprocessing respectively where the subword is converted into normalized text. Similar to how it works with BPE, sentence piece achieves higher quality tokenization and reduces error when working with longer strings and characters. It also trains non-space languages like Chinese or Japanese with the same ease as with English and French.

Conclusion

Now that we have seen how text data can be converted to tensors and also established an understanding of how different tokenizers work, in the last section of the article we’ll finally have a look at how models learn and see how different parameters influence the model’s performance.

If you missed the first part of this article, you can find it here.

Resources

Originally published at https://thulieblack.hashnode.dev on November 16, 2021.

What are tensors

Tensors are data structures that comprise of scalars, vectors, and matrices of higher dimensions. A single-dimensional tensor can be represented as a vector while a two-dimensional tensor can be represented as a matrix (see the image below).

source: Deep Learning with PyTorch

Even though we can refer to tensors as a multi-dimensional matrix, it is important to note that tensors are dynamic. This means that they have the ability to transform when performing mathematical operations.

Why Tensors Not Numpy Arrays

Numpy is a python library that performs advanced mathematical computations and is very popular in providing multidimensional arrays. Although tensors operate the same way as NumPy arrays, tensors have the ability to perform faster computations on graphical processing units(GPU) becoming the most commonly used data structure in Deep Learning. Another advantage of tensors is that they are immutable, meaning that they can be implemented in several different ways without any change in behavior. Tensors also provide low-level implementations on numerical data structures, which boosts performance and efficiency, unlike Numpy arrays.

Dimensions

Before we proceed to talk about how tensors are stored in memory, we have mentioned that tensors are multidimensional matrices. This leads to the question of what is a dimension. Basically, a dimension is an aspect/detail of something. The purpose of dimensions is to give in-depth details of our data and the more data we have the better but just as Mark Twain said too much of anything is bad, this also applies to our case as well. Let’s take for instance a greyscale image, which is represented by a collection of scalars arranged in a grid with height and width(pixels) this automatically equates to a two-dimension H + W. If we add an RGB (Red, Blue, and Green) color channel to our dimension H + W + C, this adds more detail to the image and provides the necessary data to be captured but if we add another 4th dimension, the very essence of the image becomes vague. ln as much as we connote the fact that the more dimensions we have, the more data captured, it is therefore imperative to be able to derive details that are deemed necessary to avoid wastage of resources. ln, a nutshell dimensions play a pivotal role when it comes to tensors and data as a whole.

Storage

Each tensor has a storage property assigned to it that holds data. Let’s look at what storage is and how tensors are stored. Storage is a one-dimensional array that contains values of any given data type. For values to be indexed in storage, they have to be allocated in contiguous chunks. This means that the neighboring values/elements are next to each other in memory. Since we mostly work with large dimensional data, how then do we store multi-dimensional matrices in memory when storage is only in one dimension? There are various ways to tackle this paradox but one simple way to approach this is by using Stride. Stride is an indexing method that indicates the number of elements that should be skipped over in storage in order to get to the next element of a given dimension. That’s a lot to take in but let’s take an example whereby we have a 3 by 3 tensor. We go to the first index on the first row and get all the elements, repeat the same process on the second row and finally the third. By the end of the process, the stride of this dimension will now be a 3 by 1 tensor. In simple terms whenever we call stride, we count the rows and collapse the columns. see the example below.

source: Deep Learning with PyTorch

Why 32-Bit floating point numbers are the preferred data type for tensors

There are different data types used in tensors, mainly integers, and floating points. Now that we have seen how tensors are stored, let’s talk about why use floating point numbers more than integers. Deep Learning involves processing thousands and hundreds of dimensional data, imagine the amount of computational time and memory needed to store all this information. This is why Pytorch tensors have a default data type of 32-bit float which take less computing time and less memory storage. Using these floating numbers improves model accuracy, and increases performance and efficiency. Whenever you hear about deep learning, think of floating point tensors.

Batch Size

Batch size is a hyperparameter in machine learning that consists of a number of samples(tensors) that will be passed through the network at an iteration. Let’s imagine we have a plate of rice, do we eat the rice in one go? that’s almost impossible unless you want to have an upset stomach, the same thing applies to our model we need to feed the data in bits in order to obtain good results. The batch size determines how the model performs and can be utilized by trying different sizes depending on the data size. Let’s take for instance we have 10 000 samples of data, we can define the batch size to 100, data will then be grouped and passed to the model as sets of 100. we will later see the impact of batch size

Conclusion

In the next article, we will delve into ways we can convert text data to tensors and also look at some of the most popular tokenization algorithms used in deep learning.

Resources

Originally published at https://thulieblack.hashnode.dev on November 8, 2021.

A recommender system is a subset/class of information filtering system that anticipates the evaluation or preference that a user would feed to an item, in simple terms recommender systems are software tools used to provide suggestions to a user according to their requirements.

Recommendation systems use different techniques but we are going to mainly focus on the two most popular used filtering systems.

Collaborative Filtering

Collaborative filtering (CF) is the most commonly used technique in designing recommender systems. Most popular websites like Netflix, IMDb, and Amazon make use of collaborative filtering. In this technique, collaborative filtering establishes its method by gathering data and examining information based on a user to make predictions by looking at similarities with other users.

Different types of Collaborative Filtering :

• Memory Based Filtering — this technique makes use of user rating information to anticipate the similarities between the users and items to make predictions/recommendations.
• Model-Based Filtering — in this technique we create a model by extracting information from the rating dataset (using data mining) and use the model to make recommendations/predictions.

Content-Based Filtering

Content-based filtering(CBF) algorithms recommend items to a user based on similarity count. The best matching items are recommended by comparing various items which a user previously rated before. For example, if I go to an online store and like a particular android phone, the content-based filtering algorithms will recommend pages related to android devices.

Getting Started

Now that we have laid the foundation of understanding recommender systems, let’s delve into the things needed to get started in building a recommendation system collaborative filtering model.

Data

Data is the most important must-have tool when it comes to building applications in deep learning. In this particular case, we’ll need data that contains a set of items and users that have already rated or reacted to these items. The ratings can range from 1 to 5 showing the most liked or disliked product. When working with this type of data, it will mostly form a matrix whereby each row would contain ratings given by a user and each column would contain ratings received for each item.

(ratings matrix)

In the example shown above, we have five users who have rated five items. Some cells in the matrix don’t contain any information as it is unlikely for users to rate or react to every item they come across. These particular empty cells are called sparse and the mostly filled cells are called dense.

Steps Involved

To build a system that can automatically recommend items to users based on their preferences, the first step is to find similar items and users. The next step is to predict the item ratings that a particular user has not rated.

To get started, we need to solve the following problem statements :

Approach

1. Determine which users or items are similar to one another.
2. Discover the similarities between the users or items and establish the rating a user would give to an item based on the ratings similar to other users.
3. Measure the accuracy of the ratings that were calculated.

• Because collaborative filtering contains different types of algorithms, there are multiple ways to find similarities between users and items or multiple ways to calculate ratings based on similar users. Depending on the preferred approach used, the end result would still be a collaborative filtering method.
• One crucial thing to bear in mind, in the collaborative filtering approach the similarities are not calculated by using factors such as the age of users or genre of movies but are strictly calculated on a rating basis (either implicit or explicit) a user gives to an item.
• When approaching to solve the measure of accuracy from your predictions, there are multiple techniques used like error calculation.

Error Calculating Techniques

One of the approaches used to measure the accuracy of your results is the Root Mean Squared Error(RMSE). In this technique, you predict ratings from a test dataset of user-item pairs whose ratings values are known(available). The difference between the known values and the predicted values would now be the error. We square all the error values from the test set, find the average(or mean) and take the square root of that average to get the RMSE. Another metric used to measure accuracy is the Mean Absolute Error(MAE). In this technique, we find the magnitude of error by finding its absolute value and taking the average of all error values.

Conclusion

Although there are several other recommender systems like demographic and hybrid filtering, it is very important to know that they all come with pros and cons. With that in mind, purpose and fulfillment are derived from the fact that the systems improve mankind’s way of living by removing the burdens of principle time wastage. Users are in a better position to focus on other necessities as the recommender system enables them to do so with just one search click touch of a button.

Resources

1. developers.google.com/machine-learning/reco..

2. builtin.com/data-science/collaborative-filt..

3. towardsdatascience.com/essentials-of-recomm..

4. medium.com/analytics-vidhya/movie-recommend..

Originally published at https://thulieblack.hashnode.dev on October 22, 2021.

An article comprises of the 4 aspects which l personally call “The 4Ps of the writing process”
These are :

1. The planning process
2. The Pen to paper process
3. The pruning process
4. The polishing process.

Planning process

As a writer, blogger, or even business person it’s very important to note that information is a hub of any article that you intend to tackle. Planning consists of up to 60% of any article, this implies that more energy is put into this process as it involves a lot of research and strategies on how best to achieve what you want to say. You need to be clear on what you want to write to avoid wasting time when drafting your article. Questions like the targeted audience and the timeline are also devised in this phase.

Pen-to-paper process

Writing is a process and at this stage, the then gathered information will now be put into an articulated, desired writing style. Your message has to be clear, precise, and consistent at this stage. It’s like building, a correct mixture or ratio of materials has to put in order for the structure to stand. You are basically framing the route, and the rhythm of the article taking into consideration the audience that are going to read the article. You still have room to add more information and determine the type of images to be placed as well. The process of progressive proofreading is initiated and the reason l said it’s progressive is that errors and omissions are expected in this stage.

Pruning process

Proofreading can never be overemphasized as this is one of the vital tools in writing. In big establishments, there are people who are employed specifically for this role and are majorly called editors. At this stage, you will be looking at your article with the eye, mostly the critical constructive one. Put yourself in the shoes of the readers and always remember you are your first critique. You can never lie to yourself though most people choose to ignore it. Grammar and punctuations are some of the things to look at. Take your time and if you can have a second opinion, please let them have a look and provide feedback. This will not only help you writing-wise but also improve your confidence and ultimately build networks. We can never live forever in isolation, we need each other one way or the other. ‌

Polishing Process

This is the final stage in your writing process and hence the most important one. At this stage, you leave no stone unturned. Ensuring that the article written is in sync with the heading and composition thereof is very important at this point. Final adjustments including the right images, and quotes are also the main priority as they determine how the article will be framed. Be proud of what you have archived at this stage and take it as a win for yourself, let your creative mind be enjoyed by your desired audience.

Conclusion

Writing is a skill and but not only is it a skill, it is also an engine that drives the way people think in almost every aspect of our lives. Being able to translate your imagination into pen and paper, and allowing your audience to enter your secret place through writing is such a wonderful opportunity that only open-minded writers can appreciate. Remember to stay positive, your first article might not be perfect but the more you write, the more perfect your skills will be.

gif credits Bing

Originally published at https://thulieblack.hashnode.dev on July 15, 2021.