We — SouL LLM Brews — participated in “Prompt Engineering Hackathon for Humanities.” and won the 1st runner-up prize. Cheers! Here is a summary of what happened during the hackathon.

👨🏻💻 My Take on the Hackathon

This hackathon — truly one of a kind — demanded a creative mindset to solve problems. Unlike other technical hackathons with clear technical problem statement, coherent sets of possible approaches, and proper metrics to evaluate solution, It challenged the developer (me) to become an artist and go beyond technical constraints.

🧩 Problem Statement

Through AI persona, explore LLM’s capabilities, boundaries, and it’s subtle contextual understanding, to create innovative collaboration between humans and machines. In layman’s terms, the journey of knowing a newly joined team member and finding their strengths and areas of improvement.

🔰 The Pilot

As we started with a whiteboard session, a thought brewed in my mind: How about our newly joined AI actor — LLM Brews — become a “story writer” and help us write an extended story for “Gunther” character in post-F.R.I.E.N.D.S world.

We began our expedition in prompt engineering land. We interacted and conversed with different ChatBot/AI tools to know which LLM caters to the best storytelling needs. Like ChatGPT, Bing Chat, Google Bard , Jasper.ai, Writesonic, etc.

🛠️ Pro Tips for Prompting

I like to highlight certain points that helped in the above task:

• Tweak LLM parameters: higher temperature for creativity and diversity. lower for precision
• Check LLM’s training dataset for intuition behind expected responses. Insights into crafting question and jailbreaking would tremendously helps.
• Experiment with different ChatBots for diverse response in regard to quality, style or tone.
• Use custom instructions or system prompts to guide LLM’s understanding. It shapes how the model interprets and responds to user requests. Check my custom instructions for a story writer persona.
• Boost accuracy by providing examples of desired requests and responses to LLMs.

🚦 LLM’s Limitations and Challenges

After doing a lot of back and forth with different LLMs and trying creative prompt hacks, I discovered some limitations for our story writer persona:

• Narrative Flow: Inconsistent dialogues and logical gaps disrupt the story’s flow as LLM starts to forget the crux of the previous context. Plus, LLM showed a Minimal In-Context Reasoning.
• Simplicity Over Appeal: LLM generates creative narrative, but tends to create overly simplistic stories lacking appeal and intricate plots. Unable to find sensible segue between two snippets of story.
• Emotional Depth: Struggles to capture nuanced emotions and lacks depth in character portrayal.
• Hallucinations: It said Gunther had a Ph.D. in art history, but the truth is, James Tyler, the actor playing Gunther, holds the Ph.D., not the character.

🛣️ Approaches To Tackle Limitations

In our efforts to enhance story creation, we’ve taken a two-fold approach.

• Firstly, we aimed for more focused and productive discussions, steering clear of vague and off-topic exchanges. To achieve this, we introduced two new AI personas — an insightful Critic and a keen Book Reader. Together with our core Story Writer AI, we formed a chat room involving three AI personas and three humans (us), each contributing diverse perspectives. My friend Rohan Awhad played a key role by developing a script that allows these personas to contribute by looking at the previous context. Check here prompt for different AI persona created by Amey Bhilegaonkar.
• Secondly, to enhance nuanced emotions in the narrative and depth of characters. I experimented with GenAI models — specifically, Runway, Midjourney , and DALL-E Open Ai — by passing LLM’s responses. These models, when provided with precise context, yielded appealing results. Like LLMs, these genAI models also had their own limitations.
• We considered generating human voice to express emotions to boost storytelling, but due to time constraints, we had to keep it aside.

💪 LLM Capabilities

• LLMs excel at finding small details, spotting story discrepancies, creating short story snippets, and uncovering relevant resources — though review is essential. For instance, I discovered Gunther’s fluency in Dutch and interest in art history.
• This fine details could help you as a writer aiding in a long complicated and multi-step refinement process for your drafts until the final version.

We presented our ideas/work, in a humorous way, and narrated a story — took our protagonist character Gunther and created his story using AI.

Our presentation: https://lnkd.in/gBedrYTX

💡 Final Thoughts

What this hackathon made me realize is that LLMs are not like any other technology that has deterministic outcomes. They are more like person who you don’t know, but you know the idea of person.

Now, my conclusions might be a bit blurry (Come on, we had 17 hours hustle time with just 4 hours of sleep!). If you spot any quirks or have differing views, drop a comment. Your feedback is more precious than a simple like!

Ever wonder how the warranty on the product gets calculated? and how do product manufacturers know when the product is going to fail? OR

When someone asks you about how far is that city or hotel from here and if you are unsure, why do you feel more confident & certain about telling distance in between ranges rather than guessing the exact distance?

In many cases, it’s essential to estimate value in ranges. It gives a sense of reliability and space to work on even though you don’t know the exact thing.

So let’s ponder upon how can we tell our confidence in numbers? and how can we calculate these ranges?

But … First, we need to understand point estimates.

What is a point estimate?

• In many cases, you may don’t know what is the population means. (population means is like taking average/mean height of every person in-universe. There would be people with different heights)
• Then to mimic population mean approximately, we take sample means (X̄). (sample means is like you consider the heights of only a few people around you and calculate the average/mean.)
• using X̄ we can estimate the parameter of population means. (like using central limit theorem)
• In this scenario, X̄ is the point estimate.
• the sample mean is a point estimate of the population means.

The point estimate does not reveal the uncertainty associated with the estimate.

• e.g., It is also like guessing the exact distance of the city or hotel from here without confidence.
• you are not getting a sense of how far sample means might be from population means.

There were Confidence intervals shines!!!

Introduction

• Confidence interval expresses a range of values within which we are pretty sure the population parameter lies. (how much sure? can be answered by confidence level)
• It communicates how accurate our estimate is likely to be.
• All estimates of population parameters, such as means, median, differences of means, and differences of medians should be expressed as confidence intervals.

Please note that, Here, I’m only talking about the “Mean” parameter to keep things simple.

• E.g. A 95% confidence interval says that for every 100 confidence intervals we calculate from sample data, about 95 of them will contain the population parameter and 5 will not.
• 95% confidence interval is just an interval that covers 95% of the means of all samples.

But still Why is confidence interval needed? what problem does confidence interval solve?

Why Confidence Interval?

1. There always will be sampling errors

When you take samples from the population and took their means, you will never be sure that sample means is equal to population means. This is called Sampling Error or Variation due to sampling. confidence interval tries to tackle this problem.

2. Confidence intervals measure the degree of uncertainty or certainty in sample variables. In many cases, you may don’t know what the population means. it’s difficult. So we can select different samples randomly from the same population. And computes a confidence interval for each sample to see how it may represent the true value of the population.

3. When we express an estimate of the population parameter, it will be good if you tell an estimate between ranges, rather than the exact value. e.g. it is better to tell “usually laptop prices can be between 40k to 60k”, instead of telling the price is 52k.

What is a Confidence Level?

• It refers to the percentage of probability, or certainty, that the confidence interval would contain the true population parameter when you draw a random sample many times.
• usage of it that “we are 99% certain (confidence level) that true population parameter most of these samples will lie between “this-that” intervals(confidence intervals).”
• It’s like having a bigger butterfly catching net, the more confident you are to catch butterflies.

Confidence interval and confidence level are interrelated but are not exactly the same.

Formula And Calculations

Let’s understand the formula

1. Standard Error

• The Standard Error is a measure of how is spread out we would expect those means to be

But why the Square root of “n”?

The more information we have, the less new information we get from one more observation in the sample. e.g. when you increase the sample size from 10 to 20, (the decrease in the confidence interval) or (new information you get) is far greater than increasing the sample size from 110 to 120. That’s why it makes sense to add square root to the sample size.

2. t - Confidence level value

• t value you can get from a t distribution.
• We can also use the Z value instead of the t value.
• In the above formula, if we increase the t, our confidence interval also is going to increase.
• Here are some common t values for different sample sizes and confidence levels.

3. The Margin of Error

4. Example

• if the margin of error = 10
• the sample means(X̄) = 100
• sample size(n) = 30
• t value = 2.045 (look into above table)

Then we are 95% confident that the population means lies between [90, 110] intervals.

Misconception

• let’s estimate the mean weight of apples in the orchard (really really large garden of fruits). We will take a sample of 15 apples in the orchard.

• Use the above formula to calculate the confidence interval for 95%.
• the result would be [146.7gm, 151.9gm]
• One might erroneously interpret the aforementioned 95% confidence interval of 146.7 to 151.9 grams as indicating that 95% of the data in a random sample falls between these numbers.
• We can be 95% sure that the mean weight of the apples in the orchard is somewhere between 146.7 and 151.9 grams.

This is NOT the interval that holds the weights of 95% of the apples in the orchard. It is the interval that we are 95% confident will contain the true unknown value of the population mean.

Confidence interval does not tell the percentage of data that falls between lower and upper bound, but CERTAINTY that range will contain the population means.

Example

• Assume the interval is between 72 inches and 76 inches. If the researchers take 100 random samples from the population of high school basketball players as a whole, the mean should fall between 72 and 76 inches in 95 of those samples.
• If the researchers want even greater confidence, they can expand the interval to 99% confidence. Doing so invariably creates a broader range, as it makes room for a greater number of sample means. If they establish the 99% confidence interval as being between 70 inches and 78 inches, they can expect 99 of 100 samples evaluated to contain a mean value between these numbers.

What affects the width of confidence intervals

Variation in Population

1. It’s Variation within the population of interest.
2. Population with low variations leads to similar samples with low variation, which leads to a narrow confidence interval.
3. Population with lots of variations leads to varied samples with high variation leads to wider confidence interval

Sample Size

1. It’s the number of data points in a single sample group.
2. Small samples vary more from each other and have less information, which leads to wider confidence intervals
3. Large samples are more similar to each other and have more information, which leads to narrow confidence intervals.

Calculating Confidence Intervals

1. Traditional Normal-based

• Here we specify a level of confidence.
• The more confident(t) we wish to be, the larger our confidence interval will be.
• Traditional normal-based methods are Informal, Bootstrapping, T-test, etc.

Any form of feedback or suggestion is really welcome on how I can improve this blog to be more understandable and enjoyable.

I use VS Code a lot!!! I do All kinds of developments, and projects on VS Code. When it comes to deep learning, and running heavy neural network models, my laptop is unable to wield the great power of GPU. So I have to switch to another popular option — Google Colaboratory. But this switching back and forth, make me wonder, how I can combine both and leverage the infinite power of both.

You will see many articles on opening the VSCode interface in a web browser that uses google colab. But here we taking a different road here.

So our approach is this, our code will reside in a Google Drive (as a local server)or GitHub repo(as a remote server). We will access those codes through the remote server option available through VS Code’s extensions. This means we will use compute power of google colab to run our code in git or google drive.

But What is VS Code?

Visual Studio Code (often abbreviated as “VS Code”) is a free, open-source code editor developed by Microsoft. It is designed to be lightweight, fast, and customizable, with a wide range of features that make it popular among developers. VS Code includes support for dozens of programming languages, as well as features like debugging, version control, and extensions that can be used to customize the editor to meet individual needs. It also includes built-in support for Git, IntelliSense code completion, and a powerful command-line interface. Overall, VS Code is a powerful, versatile tool for editing and managing code that is widely used by developers around the world.

And what is Google Colab?

Google Colab (short for “Collaboratory”) is a free cloud-based platform that provides an interactive environment for writing, running, and sharing Jupyter notebooks. It allows users to write and execute Python code, including data analysis and machine learning tasks, without the need for specialized hardware or software. Colab provides access to powerful computing resources, including GPU and TPU acceleration, and allows users to share their work with others through Google Drive. The platform is widely used by students, researchers, and developers for a wide range of applications, from data exploration and visualization to deep learning and AI research.

Steps:

• Open google drive where you will clone your GitHub repo.
• Choose an apt folder for your Repo in google drive.
• Create a Colab notebook there. start writing the following code in it.
• Mount your drive in that colab. (let colab know where we want to clone our repo)

NOTE: You must turn the GPU option “ON” when you run this colab notebook. Because compute power of this colab notebook is going to share with the remote you opening in VS code.

# mount the google drive path
from google.colab import drive
drive.mount('/content/drive')
# folder path where you want to keep your repo
%cd drive/My Drive/Colab Notebooks

If your repository is private.

!git clone <https://<YOUR_USERNAME>:<PAT_TOKEN>@github.com/><USERNAME_OF_REPO_OWNER>/REPO_NAME

!git clone https://Ztrimus:pat_token@github.com/OmdenaAI/Dryad
!cd Dryad

Then install Colab-ssh library

# Install colab_ssh on google colab
!pip install colab_ssh --upgrade

Then run the following code

from colab_ssh import launch_ssh_cloudflared, init_git_cloudflared
launch_ssh_cloudflared(password="enter_password_here")

After running it you will see

If it’s your first time click on Client Machine Configuration.

1. got to your VS code.
2. Install an Extension called Remote-SSH
3. Then press Ctrl+Shift+P. type “Remote-SSH: Open Configuration File…”

Then Select configuration file or create a new file

Then paste the code which you saw in the client machine configuration output

Now you are done with your configuration of colab ssh in your local VS Code. The only thing remaining is to start your remote SSH. Copy your VSCode Remote SSH from colab notebook

Go to your VS Code and press Ctrl+Shift+P and Type “Remote-SSH: Connect to Host…”

Paste your VSCode Remote SSH in it and hit enter Then Vola!!!!!!

probably it’s going to ask you about Which OS your want to choose.

Enter the password you set above while configuring the remote in colab notebook.

It will look something like this. you can see your SSH at the lower bottom in green.

1. Now it’s connected to your Colab machine with Drive mounted on it (Yeah! it’s an important one)
2. Now go to Explorer (or press Ctrl+Shift+E) and Press open folder.

1. Replace /root/ with /content/drive + drive_folder_path where your GitHub repo is stored

Then done !!!

you can use colab notebook resources directly in VS code. You can use GPU as well.

To run python install the Python extension from VS Code.

So, That’s it, guys!!!
Please let me know any improvements and feedback you feel are going to make this article more useful and easy for others to understand and comprehend.