Delivered a 3-hour Docker 101 workshop on container-supported development workflow

GovTech STACK Conference 2024: A Unique Developer Event

GovTech Singapore’s STACK Developer Conference is unlike any other developer event I’ve attended. It’s one of Singapore’s largest tech conferences and one of the few government-led developer events worldwide. This conference showcases Singapore’s digital capabilities and highlights the pivotal role of developers in Singapore’s journey toward becoming a digital government.

Singapore’s Vision for a Digital Government

The STACK 2024 keynote speakers set the stage by emphasizing the critical role developers play in building Singapore’s digital government. This year, Singapore ranked third globally in the United Nations E-Government Survey, a testament to its commitment to digital transformation. The conference also provided a sneak peek into Singapore’s Smart Nation 2.0 vision. This initiative focuses on creating a digitally trusted society, boosting economic growth, and building stronger communities, all while tackling new online challenges.

Pioneering AI in Government

Another exciting theme at STACK 2024 was Singapore’s push to integrate AI into government functions. In his opening speech, GovTech Singapore’s Chief Executive, Goh Wei Boon, discussed their ambitious plans to introduce AI-powered code assistants for government developers, aimed at boosting productivity. This aligns with recent findings from Stack Overflow’s 2024 survey, which shows 72% of developers view AI coding tools positively and don’t see them as a threat to their jobs.

The Singapore Tech Stack: Building Fast, Building Smart

One of the standout features of Singapore’s digital government strategy is the Singapore Tech Stack — a set of reusable components that helps developers build applications quickly and efficiently. A great example is MyInfo, a secure service that allows residents to access and manage personal data and auto-fill forms. Thanks to the Tech Stack, MyInfo was developed in just four months. Another example is GovWallet, which provides a seamless experience for residents to access government payouts across different platforms.

At the heart of this tech ecosystem is the Government on Commercial Cloud (GCC), which enables the government to adapt and scale rapidly. Singapore has already migrated 80% of eligible systems to GCC, a remarkable achievement that makes it one of the most successful government cloud migrations globally.

Exploring GovTech’s Product Booths

After my Docker workshop for around 40 attendees, I had the chance to explore some fascinating booths showcasing GovTech’s products. Here are some of the highlights:

Government on Commerical Cloud

GCC is a platform that helps Singapore government agencies quickly and securely adopt commercial cloud solutions like AWS, Microsoft Azure, and Google Cloud. GCC 2.0 has streamlined onboarding that lets users get started within an hour through their TechPass account.

GCC 2.0 adopts a Policy-as-Code (PaC) approach, ensuring that all resources undergo automatic compliance checks by default. Compliance checks are performed in real-time to identify potential security vulnerabilities as soon as they arise.

This version of GCC emphasizes a “light-touch” approach for cloud-native solutions, using native tools like AWS Config to maintain continuous compliance. By embedding policies as code, GCC 2.0 minimizes latency in security assessments, allowing faster evaluation of cloud workloads’ compliance states and early detection of configuration changes. This proactive approach supports secure and efficient cloud operations for government agencies.

AISAY

AISAY is an AI document reader capable of extracting information from various document types — handwritten notes, printed articles, and more. Unlike traditional OCR, AISAY handles unstructured data, multilingual text, and images, providing structured responses for documents from local entities.

GovText

GovText is a text analytics tool that uses NLP to help government agencies analyze and interpret unstructured data, improving data processing and operational efficiency. The GovText Web Portal offers two types of summarisation:

• Abstractive summarization (Normal mode) — The key points of an article are paraphrased into a short and coherent paragraph, by a language model pre-trained on news articles and open datasets.
• Extractive summarization (Quick mode) — The most important sentences from an article are compiled to form a summary. While this approach does not involve paraphrasing, it offers faster processing speed.

CloudSCAPE

This platform helps government agencies maintain security compliance for systems hosted on GCC. It includes automatic scans to alert administrators when security baselines aren’t met.

With over 70 agencies onboarded, this platform provides automated daily scanning of GCC 2.0 cloud resources for compliance with Singapore Government IM8.

CStack

CStack integrates with agile development tools to make it easier for teams to modernize applications without worrying about infrastructure complexities. It supports deployment on both internet and intranet environments.

CStack meets the Singapore Government’s Application Infrastructure Architecture Standard (AIAS) and IM8 requirements. It utilises a default tiered architecture (web, app, data) and adheres to relevant policies for infrastructure, applications, and containers. By using CStack, you will streamline compliance audits for these areas and save time in the process.

With CStack, you can deploy your applications on either the internet or intranet. Using CStack GEN routable clusters, you can enable end users to access your applications by whitelisting your egress firewalls to the platform ingress.

Sentinel

I simply love this booth. Sentinel is a collection of LLM guardrail filters to detect unsafe or irrelevant content. With Sentinel, developers can block or adjust such content before it reaches the LLM or is returned to the users.

Docker DevTools : GenAI Meetup at AWS Office

This event marked a significant milestone for me as it was my first time conducting an event in Singapore. Having no prior knowledge of the local developer community or the AI landscape in the region, I was eager to dive in and connect with fellow tech enthusiasts.

Thanks to Amazon Web Services (AWS) for providing such a wonderful venue, we were able to host a highly engaging and interactive session. The event brought together developers and AI experts to explore the latest innovations in generative AI, Docker, and cloud technologies.

Chomchana (Chom) Trevai, Ph.D. kickstarted the event and talked about Building Serverless Generative AI application on AWS.

Next, I delivered a talk on Build and Deploy AI-powered Apps locally using GenAI Stack. Showcased a couple of demo on building a Streamlit application using LangChain, Neo4j and Ollama.

The next speaker, Elliott Chong , an AI/ML Hackathon winner talked about the recent enhancement with Dionysus — A Powerful Platform for Streamlined Developer Collaboration.

Next, Ivan Tang delivered his talk on Milvus and Zilliz Cloud. Milvus is a cloud-native vector database for next-generation AI applications.

Finally, Michael Yuan joined us for US virtually and talked for 25 min on a title “Portable AI application in Docker across CPUs and GPU”.

It was an exciting experience to see so many enthusiastic participants come together, share ideas, and learn about building serverless AI applications, deploying AI-powered apps locally, and exploring the future of AI with Docker. The diverse discussions, coupled with valuable insights from industry leaders, made this event truly special.

I’m grateful for the opportunity to be part of this vibrant tech community in Singapore, and I look forward to more such events where we can continue to learn, collaborate, and innovate together!

A Memorable Conference Experience

GovTech Singapore STACK 2024 was a surge of new knowledge, networking, and exploration. From the fourteen product showcases and over 40 lightning talks to keynotes and panel discussions, it was packed with opportunities to learn and connect. I left Singapore with a deeper understanding of its tech landscape and a strong appreciation for the innovative work happening in the government sector.

If you ever get the chance to attend STACK, don’t miss it! It’s an event that truly brings together technology, innovation, and community in a way that’s uniquely Singapore.

Welcome to Collabnix Connect, a platform designed to bring DevOps professionals together in one place to share knowledge, collaborate on projects, and grow through tech meetups. We’re super thrilled to launch this community-driven hub, built with the goal of fostering learning, networking, and support within the global DevOps community.

What is Collabnix Connect?

Collabnix Connect is a platform where DevOps practitioners, engineers, and enthusiasts can:

• Share their knowledge: Post articles, tutorials, and insights on the latest DevOps tools, trends, and best practices.
• Join technical Meetup groups: Find and join technical communities based on your interests, participate in meetups, and engage with like-minded individuals.
• Stay updated: Share your updates on projects, events, or any significant learnings to keep the community informed and inspired.
• Collaborate and grow: Whether you’re new to DevOps or a seasoned expert, Collabnix Connect encourages open discussions, collaborations, and contributions.

Why Collabnix Connect?

As DevOps continues to evolve and shape the future of software development and IT operations, the need for a dedicated platform to bring professionals together has never been greater. Collabnix Connect aims to be that bridge — helping you find the right resources, the right people, and the right opportunities to advance your skills and knowledge.

Here’s what you can expect from our platform:

• Technical Tutorials: In-depth guides on popular DevOps tools like Docker, Kubernetes, CI/CD pipelines, and more.
• Case Studies: Real-world stories and best practices from DevOps teams across the globe.
• Event Highlights: Recaps from DevOps meetups and conferences, with takeaways and insights from industry leaders.
• Tool Reviews: Unbiased reviews of the latest DevOps tools to help you make informed decisions.
• Community Stories: Hear from other DevOps professionals about their journey, challenges, and successes.

Join Us on This Journey

We’re just getting started, and we want YOU to be part of this community. Here’s how you can get involved:

• Subscribe: Stay up-to-date with our latest blog posts, news, and events.
• Contribute: We’re always looking for guest contributors! If you have knowledge to share or a story to tell, reach out to us to feature your post on Collabnix Connect.
• Engage: Join discussions, participate in meetups, and share your feedback on how we can improve and grow this community.

A Call to Action for DevOps Professionals

Whether you’re a developer, engineer, architect, or just someone passionate about DevOps, Collabnix Connect is your space to explore, learn, and connect. Our mission is simple: to build a thriving community where learning never stops, and collaboration leads to innovation.

Thank you for being part of this journey with us. We look forward to sharing, learning, and growing with you. Stay tuned for exciting updates, insightful content, and much more!

Let’s build something amazing together!
Visit the platform: https://connect.collabnix.com

Follow us on Social Media:

Twitter: https://x.com/collabnix
LinkedIn: https://www.linkedin.com/company/collabnix/
GitHub: https://github.com/collabnix

Docker has become an indispensable tool for developers to package and deploy applications. A crucial aspect of efficient Docker development is ensuring the correctness of your Dockerfile. This is where Docker Build Checks come into play.

Typically, when you run a build, Docker executes the build steps in your Dockerfile and build options as specified. With build checks, rather than executing the build steps, Docker checks the Dockerfile and options you provide and reports any issues it detects.

Is Docker Build Check a new feature?

Yes. Docker Build Checks is a feature introduced in Dockerfile 1.8 that allows you to validate your build configuration without actually executing the build. By running docker build --check ., you can identify potential issues in your Dockerfile early in the development process, saving time and effort.

How does it work?

Checks run as a build invocation, but instead of producing a build output, it performs a series of checks to validate that your build doesn’t violate any of the rules.

You can use the --check flag with the docker build or docker buildx build command to run the checks.

For Example,

$ docker build --check .

This command will run the checks and report any issues it finds without executing the build steps. If there are any issues, they will be reported in the output.

Sample App: Flask + Python

The provided Python script creates a basic web application that displays the number of times it has been accessed. It utilizes Flask for the web framework and Redis to store the visit count.

How it Works?

When someone accesses the root URL (e.g., http://localhost:5000), the hello function is triggered. The function increments the ‘hits’ counter in Redis. It then retrieves the current value of ‘hits’ and displays it in a user-friendly message.

File: app.py

cat app.py 
from flask import Flask
from redis import Redis
import os
import socket

app = Flask(__name__)
redis = Redis(host=os.environ.get('REDIS_HOST', '127.0.0.1'), port=6379)

@app.route('/')
def hello():
    redis.incr('hits')
    return f"This webpage has viewed {redis.get('hits').decode('utf-8')} times and hostname is {socket.gethostname()}.\n"

Let’s create a Dockerfile for this app.py

FROM python:3.11-slim

RUN pip3 install flask redis && \
    groupadd -r demouser && useradd -r -g demouser demouser && \
    mkdir /src && \
    chown -R demouser:demouser /src

USER demouser

COPY app.py /src/app.py

WORKDIR src

ENV FLASK_APP=app.py REDIS_HOST=redis

EXPOSE 5000

CMD ["flask", "run", "-h", "0.0.0.0"]

Let’s try to run docker build --check

[internal] load build definition from Dockerfile                            0.0s
 => => transferring dockerfile: 398B                                            0.0s
 => [internal] load metadata for docker.io/library/python:3.11-slim             2.8s
 => [auth] library/python:pull token for registry-1.docker.io                   0.0s
 => [internal] load .dockerignore                                               0.0s
 => => transferring context: 2B                                                 0.0s

WARNING: WorkdirRelativePath - https://docs.docker.com/go/dockerfile/rule/workdir-relative-path/
Relative workdir "src" can have unexpected results if the base image changes
Dockerfile:12
--------------------
  10 |     COPY app.py /src/app.py
  11 |     
  12 | >>> WORKDIR src
  13 |     
  14 |     ENV FLASK_APP=app.py REDIS_HOST=redis
--------------------

The issue is clearly related to https://docs.docker.com/reference/build-checks/workdir-relative-path/

Modify the following line in your Dockerfile to:

WORKDIR /src

docker build --check .
[+] Building 3.2s (4/4) FINISHED                                docker:desktop-linux
 => [internal] load build definition from Dockerfile                            0.0s
 => => transferring dockerfile: 399B                                            0.0s
 => [internal] load metadata for docker.io/library/python:3.11-slim             3.1s
 => [auth] library/python:pull token for registry-1.docker.io                   0.0s
 => [internal] load .dockerignore                                               0.0s
 => => transferring context: 2B                                                 0.0s
Check complete, no warnings found.

BuildKit has built-in support for analyzing your build configuration based on a set of pre-defined rules for enforcing Dockerfile and building best practices. Adhering to these rules helps avoid errors and ensures good readability of your Dockerfile.

Common Dockerfile Warnings

Warning CodeDescriptionstage-name-casingStage names should be lowercasefrom-as-casingThe ‘as’ keyword should match the case of the ‘from’ keywordno-empty-continuationEmpty continuation lines will become errors in a future releaseconsistent-instruction-casingAll commands within the Dockerfile should use the same casing (either upper or lower)duplicate-stage-nameStage names should be uniquereserved-stage-nameReserved words should not be used as stage namesjson-args-recommendedJSON arguments recommended for ENTRYPOINT/CMD to prevent unintended behavior related to OS signalsmaintainer-deprecatedThe MAINTAINER instruction is deprecated, use a label instead to define an image author

References:

• Docker Build Checks

Are you ready to revolutionize your applications with the power of Large Language Models (LLMs)? Join us for the Gen-AI Stack Meetup in Singapore, where we’ll explore the cutting-edge intersection of LLMs, graph databases, and Docker.

This half-day event will delve deep into the practical applications of the Gen-AI Stack, offering insights, hands-on workshops, and networking opportunities. Whether you’re a seasoned developer or just starting your AI journey, there’s something for everyone.

Key topics to be covered:

• Integrating LLMs with the Gen-AI stack for innovative applications
• Fine-tuning LLMs for specific industry needs
• Ethical considerations in LLM development
• Real-world LLM use cases and success stories
• MLOps best practices for deploying and managing LLM-powered applications

Share your expertise! We’re inviting talks from experts in the field. Submit your proposal by October 27th and share your knowledge with the community.

Event Details:

• Date: November 8, 2024
• Time: 1:00 PM to 5:00 PM
• Location: Amazon Web Services Singapore Private Limited, 23 Church Street, Capital Square, Unit 10–01 to 10–04, 049481, Singapore

Don’t miss this opportunity to learn from industry leaders, network with fellow AI enthusiasts, and explore the future of AI development.

Register now and secure your spot!

The software development landscape is undergoing a dramatic transformation, fueled by the integration of artificial intelligence (AI) and machine learning (ML). Docker, a pivotal player in the containerization ecosystem, recently released its 2024 AI Trends Report, shedding light on the increasing role AI plays in shaping the future of development.

Based on a survey of over 1,300 developers, the report provides invaluable insights into how AI is being used in development workflows, the most important AI trends, and the evolving attitudes toward AI across the industry.

Click to Access to the 2024 Docker AI Trends Report

Here are the Top 10 Takeaways from the report, highlighting the future of AI in software development.

1. Machine Learning (ML) Engineering is on the Rise

The report underscores significant growth in ML engineering and data science within the Docker ecosystem. More developers are leveraging Docker to manage and scale ML workflows, integrating AI-driven applications in containerized environments. This growth reflects a broader trend: AI isn’t just a buzzword anymore — it’s becoming a core part of development.

2. GenAI Leads the Pack of Emerging Trends

Generative AI (GenAI) is viewed as the most important trend in software development, with 40% of respondents identifying it as a key focus. GenAI’s ability to automate content creation, code generation, and even design processes is streamlining workflows for developers. Following closely behind, AI assistants for software engineering — used to assist with code, documentation, and debugging — were chosen by 38% of respondents as a vital trend to watch.

3. AI is Used Across Diverse Company Sizes

The survey paints a picture of diverse users, with 42% of respondents from small companies, 28% from mid-sized organizations, and 25% from large enterprises. This breadth showcases AI’s reach across companies of all sizes, demonstrating that AI is no longer just a tool for tech giants — it’s accessible and valuable to everyone.

4. AI is Becoming an Essential Tool in Development

AI adoption is growing rapidly, with 64% of developers reporting they use AI in their daily work. AI is being applied in a variety of areas:

• 33% use it to write code.
• 29% use it for writing documentation.
• 28% leverage AI for research purposes.

These applications are helping developers to accelerate their productivity by handling routine or complex tasks.

5. Generational Divide on AI Priorities

The survey revealed a divide in the prioritization of AI trends between senior and junior developers. Senior developers, DevOps engineers, and platform managers consider GenAI the most important trend, seeing its potential in automating complex workflows. On the other hand, junior developers place more emphasis on AI assistants for writing code and performing routine tasks. This generational difference highlights AI’s diverse benefits for developers at different stages of their careers.

6. ChatGPT and GitHub Copilot Dominate the AI Toolset

The most popular AI tools among developers reflect a trend toward leveraging AI for practical coding support. 46% of developers use ChatGPT, while 30% rely on GitHub Copilot. Additionally, 19% of developers are using Bard, indicating a growing ecosystem of AI-powered coding assistants helping developers write better, faster code.

7. Positive AI Sentiments Prevail

While concerns about AI’s role in the workplace remain, 65% of respondents agree that AI is a positive force in their work environment. More than half (61%) say AI makes their jobs easier, and 55% believe that AI enables them to focus on more important tasks by automating routine processes.

8. AI and Job Security: A Mixed Bag

Despite the overall positive sentiment, some developers express concerns about AI’s impact on job security. 23% of respondents see AI as a potential threat to their jobs, while 19% believe it makes their work more difficult. These mixed feelings reflect ongoing debates in the industry about whether AI will complement human skills or replace certain roles entirely.

9. Moderate AI Dependency Among Developers

When asked how dependent they are on AI for their work, developers rated their reliance on AI tools at an average of 4.04 out of 10. This score suggests that while AI tools are becoming essential, many developers still view them as complementary rather than indispensable. As AI tools continue to evolve, this dependency is likely to increase over time.

10. AI Experience Matters: More Years, More GenAI Focus

The report revealed that developers with more than six years of experience are more likely to prioritize GenAI in their work, whereas developers with fewer than five years of experience place greater importance on AI assistants. This suggests that experienced developers see the potential for AI to revolutionize higher-level workflows, while less-experienced developers appreciate the immediate benefits of AI-assisted code generation.

The Road Ahead for AI and Docker in 2024

The findings from Docker’s 2024 AI Trends Report make one thing clear: AI is becoming a cornerstone of software development, from writing code and documentation to streamlining research and debugging. As more developers adopt AI tools like ChatGPT, GitHub Copilot, and Bard, we can expect even faster innovation in how software is developed, tested, and deployed.

With Generative AI and AI assistants leading the charge, developers are empowered to automate routine tasks, reduce errors, and focus on building more innovative solutions. And as Docker continues to be a pivotal platform in the AI/ML ecosystem, it will play an increasingly important role in shaping the future of AI-driven software development.

As AI and ML trends evolve, developers should stay informed and explore how these technologies can enhance their workflows. The future is bright, and AI, coupled with Docker’s powerful containerization tools, will undoubtedly drive the next wave of innovation.

Stay Updated

To keep up with the latest trends in AI and Docker, subscribe to the Docker newsletter and stay tuned for the full report release. As AI transforms how we build software, staying ahead of the curve has never been more important.

Ready to dive into AI with Docker? Start by exploring Docker’s AI tools and resources, and begin containerizing your AI-driven applications today.

Check out the list of Docker AI/ML stories

Further Readings

• Getting Started with GenAI Stack powered with Docker, LangChain, Neo4j and Ollama
• Introducing a New GenAI Stack: Streamlined AI/ML Integration Made Easy
• Local LLM Messenger: Chat with GenAI on Your iPhone
• Using Generative AI to Create Runnable Markdown
• ReadMeAI: An AI-powered README Generator for Developers
• Build Your Own AI-Driven Code Analysis Chatbot for Developers with the GenAI Stack
• A Quick Guide to Containerizing Llamafile with Docker for AI Applications
• A Promising Methodology for Testing GenAI Applications in Java
• Building a Video Analysis and Transcription Chatbot with the GenAI Stack
• Build Multimodal GenAI Apps with OctoAI and Docker
• LLM Everywhere: Docker for Local and Hugging Face Hosting
• Build and Deploy a LangChain-Powered Chat App with Docker and Streamlit
• How to Run Hugging Face Models Programmatically Using Ollama and Testcontainers
• Effortlessly Build Machine Learning Apps with Hugging Face’s Docker Spaces
• Docker and Hugging Face Partner to Democratize AI
• How an AI Assistant Can Help Configure Your Project’s Git Hooks
• Docker Documentation Gets an AI-Powered Assistant
• The Strategic Imperative of AI in 2024
• “@docker can you help me…”: An Early Look at the Docker Extension for GitHub Copilot
• Streamline the Development of Real-Time AI Applications with MindsDB Docker Extension
• Creating AI-Enhanced Document Management with the GenAI Stack
• Better Debugging: How the Signal0ne Docker Extension Uses AI to Simplify Container Troubleshooting
• AI Trends Report 2024: AI’s Growing Role in Software Development
• Docker Partners with NVIDIA to Support Building and Running AI/ML Applications
• Empowering Data-Driven Development: Docker’s Collaboration with Snowflake and Docker AI Advancements
• Announcing Docker AI/ML Hackathon
• How IKEA Retail Standardizes Docker Images for Efficient Machine Learning Model Deployment
• How to Get Started with the Weaviate Vector Database on Docker
• Accelerating Machine Learning with TensorFlow.js: Using Pretrained Models and Docker
• Sentiment Analysis and Insights on Cryptocurrencies Using Docker and Containerized AI/ML Models
• Supercharging AI/ML Development with JupyterLab and Docker
• Why Are There More Than 100 Million Pull Requests for AI/ML Images on Docker Hub?
• Optimizing Deep Learning Workflows: Leveraging Stable Diffusion and Docker on WSL 2
• Conversational AI Made Easy: Developing an ML FAQ Model Demo from Scratch Using Rasa and Docker
• Full-Stack Reproducibility for AI/ML with Docker and Kaskada
• How to Develop and Deploy a Customer Churn Prediction Model Using Python, Streamlit, and Docker
• Build and Deploy a Retail Store Items Detection System Using No-Code AI Vision at the Edge
• How to Train and Deploy a Linear Regression Model Using PyTorch
• How to Deploy GPU-Accelerated Applications on Amazon ECS with Docker Compose
• Top Developer Trends for 2021
• Depend on Docker for Kubeflow

This perception — that Docker Desktop is just a graphical user interface (GUI) — is widespread, but it misses the broader picture. Docker Desktop isn’t just a UI; it’s a comprehensive development platform that integrates a suite of tools, services, trusted content, and automation. These features are designed to streamline development, enabling faster and more secure delivery of applications. In this blog, we’ll explore why Docker Desktop is so much more than just a GUI and why it has earned its place as the #1 most-used developer tool in Stack Overflow’s 2023 Developer Survey.

Docker Desktop: More Than Just a UI

The graphical interface is a core aspect of Docker Desktop, but that’s just one part of the picture. Docker Desktop offers a wide range of functionalities that go beyond the visual experience, making it an essential tool for developers who want to work efficiently, securely, and with greater flexibility.

Docker Desktop integrates a comprehensive set of tools and services, allowing developers to automate and optimize their workflows from development through deployment.

Addressing Key Developer Needs: Speed, Security, and Choice

Docker consistently ranks highly in the Stack Overflow community because it effectively addresses three key concerns: speed, security, and choice. Let’s break down how Docker Desktop tackles each one.

I. Speed

For developers, time spent on setup and configuration is time taken away from writing code. Docker Desktop helps speed up this process through various features:

1. Docker Init

Automatically generates Dockerfiles and docker-compose.yml files for popular programming languages like Python, Node, and Go, reducing the time it takes to start new projects.

Docker init is a CLI command that was introduced in Docker Desktop to simplify the process of initializing a new project to run in a Docker container. When you run the docker init command in your project directory, it will guide you through the creation of the necessary files for your project with sensible defaults. These files include:

• .dockerignore
• Dockerfile
• docker-compose.yaml

The docker init command also allows you to choose the application platform that your project uses and the relative directory of your main package. Additionally, it can generate Docker assets for your project using the Docker Scout CLI and provides an adminless Mac install flow that does not require admin privileges. Do check: Introducing Docker Init: Generating Docker Assets for Your Projects

2. VirtioFS Support

Reduces database import times by 98%, significantly speeding up local development

VirtioFS is a high-performance file system that can be used with Docker containers to improve their performance.In this post, we will explore what VirtioFS is, how it works, and the benefits of using it with Docker Desktop. Docker Desktop uses VirtioFS to mount the host file system in the container as a shared file system. This allows the container to access files on the host machine directly, without the need for a network share or a volume mount. Do check: Unlocking High Performance with VirtioFS in Docker Desktop

3. Docker Compose File Watch

Syncs local code changes with containers in real-time, eliminating manual restarts and updates.

Docker Compose Watch is a file watch command that automates the update process for running Docker Compose services as developers edit and save their code. By monitoring specified file or directory paths on the host machine, Docker Compose Watch detects changes and performs corresponding actions within the service container. This capability enables developers to instantly see the impact of their code changes without manually triggering image builds or container restarts. Do check: What is Docker Compose Watch and what problem does it solve?

4. vpnkit to gVisor Networking:

Improves container-to-host networking performance by up to 5X.

In Docker Desktop 4.19, Docker introduced a major enhancement to boost VM networking performance on macOS Ventura and later versions. By replacing VPNKit with the gVisor TCP/IP stack, this update delivers a 5x increase in container-to-host networking speed. This change significantly improves communication between containers and external servers, making it especially valuable for developers who rely on frequent network operations like downloading packages via npm install or apt-get.

This update not only optimizes container-to-host communication but also enhances overall efficiency, providing a faster, more responsive development environment on macOS. Do check: Docker Desktop 4.19: Compose v2, Moby Project Update, and New Language Support

5. Docker Build Cloud

Docker builds. 39x faster. Executable in the cloud.

Docker Build Cloud is a service that lets developers build their container images faster, both locally and in CI. Builds run on cloud infrastructure optimally dimensioned for your workloads, with no configuration required. The service uses a remote build cache, ensuring fast builds anywhere and for all team members. It solves the problem of slow build times by running builds locally and in the cloud. This means that developers can take advantage of the powerful hardware and resources that are available locally and in the cloud, which can significantly speed up the build process. Do check: Supercharge your Arm builds with Docker Build Cloud: Efficiency meets performance

II. Security

Security is a top concern for developers, especially with increasing demands for secure coding practices. Docker Desktop provides a set of security-focused tools that seamlessly integrate into the development workflow:

1. Docker Scout

Automatically scans for vulnerabilities and recommends fixes while you code, allowing you to address security concerns early in the development process

Docker Scout is a collection of software supply chain features that appear throughout Docker user interfaces and the command line interface (CLI). These features provide detailed insights into the composition and security of container images.

Docker Scout is designed with developers in mind and integrated into Docker. With Docker Scout, spend less time searching for and fixing vulnerabilities, and more time developing your code. Docker is building Docker Scout to sit as a layer on top of the Docker ecosystem to help developers build and maintain a secure software supply chain. Right now, Docker is focussed on helping with vulnerability remediation; we think our CVE-to-package matching (using PURLs to help avoid false positives) and our SBOM-to-CVEdb matching (no need to rescan) are both nice improvements to the current Developer experience. Do check: A First Look at Docker Scout — A Software Supply Chain Security for Developers

2. Attestations

Automatically generates Software Bill of Materials (SBOMs) and SLSA Provenance, ensuring that security credentials are embedded directly into your image.

Attestations in Docker describe how an image was built and what it contains. They are created at build-time by BuildKit and become attached to the final image as metadata. The purpose of attestations is to make it possible to inspect an image and see where it comes from, who created it and how, and what it contains. This enables you to make informed decisions about how an image impacts the supply chain security of your application. It also enables the use of policy engines for validating images based on policy rules you’ve defined.
Do check: Attestations

III. Choice

Developers appreciate flexibility and choice in their toolsets, and Docker Desktop delivers:

1. Docker Extensions

Allows developers to seamlessly connect their favorite development tools to your application development and deployment workflows

With over 700 Docker Extensions available on Docker Hub, Docker Desktop enables developers to customize their environment with tools tailored to their specific needs, ranging from Kubernetes development to security enhancements.

Docker Extensions lets you use third-party tools within Docker Desktop to extend its functionality. It allows developers to seamlessly connect their favorite development tools to your application development and deployment workflows. Do check: A Curated List of Docker Extensions

2. Docker-Sponsored Open Source Projects

Developers can access more than 600 open-source projects for free via Docker Hub, fostering collaboration and innovation.

Docker indeed supports open-source projects through its Docker-Sponsored Open Source Program. This program publishes and maintains Docker Open Source Images on Docker Hub. These images are part of a special program and have a special badge on Docker Hub, making it easier for users to identify projects that Docker has verified as trusted, secure, and active open-source projects.

The Docker-Sponsored Open Source (DSOS) Program provides several features and benefits to non-commercial open source developers. The program grants the following perks to eligible projects:

• Repository logo
• Verified Docker-Sponsored Open Source badge
• Insights and analytics
• Access to Docker Scout for software supply chain management
• Removal of rate limiting for developers
• Improved discoverability on Docker Hub

Do check: Docker-Sponsored Open Source Program

3. Multiple Architectures

A single docker build command can produce images compatible with different architectures, including x86, ARM, RISC-V, and even IBM mainframes, making Docker Desktop a versatile solution for cross-platform development.

Docker provides support for building images for multiple architectures. This is particularly useful when you want your applications to run on different types of hardware. The architectures that are highly recommended for your extension are linux/amd64 and linux/arm64

By default, Docker Desktop’s builder doesn’t support building multi-platform images. You can enable this feature by switching to a different driver or turning on the containerd image store.

Do check: Build multi-platform images

4. WebAssembly

The Docker Desktop Wasm feature allows you to run Wasm applications alongside your Linux containers. This means that you can use Docker to build, run, and share both Wasm and Linux applications.

To use the Docker Desktop Wasm feature, you need to enable the Wasm feature in the Docker Desktop settings. Once the Wasm feature is enabled, you can create and run Wasm containers just like you would any other Docker container.

Docker sees Wasm as a complementary technology to Linux containers. This is where developers can choose which technology they use (or both!) depending on the use case. And as the community explores what’s possible with Wasm, the effort is to make Wasm applications easier to develop, build, and run using the experience and tools you know and love. Docker solves the traditional problem — it works on my machine, works on yours and in production flawlessly. Docker Desktop and CLI can now manage both Linux containers and Wasm containers side by side.

The Wasm feature is still in Beta. We recommend that you do not use this feature in production environments as this feature may change or be removed from future releases.

List of Supported Wasm Runtimes

Docker Desktop support the following Wasm runtime:

• Slight: Slight is a lightweight Wasm runtime that is designed to be fast and efficient. It is a good choice for running Wasm workloads that require high performance.
• Spin: Spin is a serverless Wasm runtime that is designed to be easy to use and deploy. It is a good choice for running Wasm workloads that are event-driven or that need to be scaled horizontally.
• WasmEdge: WasmEdge is a full-featured Wasm runtime that supports a wide range of features. It is a good choice for running Wasm workloads that require a high degree of flexibility and control.
• Wasmtime: Wasmtime is a Wasm runtime that is designed to be compatible with the WebAssembly specification. It is a good choice for running Wasm workloads that need to be portable to different environments.

Conclusion

Docker Desktop is far more than just a GUI; it’s a comprehensive development platform that enhances your workflow with speed, security, and choice. Its continued recognition as the #1 most-used developer tool in Stack Overflow’s 2023 Developer Survey reinforces its value to the developer community.

For those who are still asking, “Why should I use Docker Desktop when I’m comfortable with the CLI?” — the answer is simple. Docker Desktop offers a robust set of tools and automations that go beyond what the CLI alone can provide. It empowers developers to work smarter, faster, and more securely, making it an indispensable tool in modern development environments.

Are you new to Kubernetes and looking to gain a solid understanding of its core concepts? Join us for a one-day workshop on Kubernetes for beginners!

In this intensive workshop, we will cover Pod101, Service101, Deployment101, Network Policy101 and much more… providing you with a comprehensive overview of these fundamental components of Kubernetes.

In this workshop, you will learn the basics of Kubernetes architecture and how to create, manage, and scale containerised applications using Pods, Services, Deployments, Network Policies etc. You will also gain hands-on experience through practical exercises and real-world examples, allowing you to apply your newly acquired knowledge in a practical setting.

Whether you are a developer, operations engineer, or IT professional, this workshop is designed to give you a solid foundation in Kubernetes, one of the most popular container orchestration platforms in the world.

No prior experience with Kubernetes is required, making this workshop perfect for beginners who are looking to dive into the world of Kubernetes and containerisation.

Workshop Topics:

— Pod101

— Service101

— Deployment101

— ReplicaSet101

— Scheduler101

— Services101

— StatefulSet101

— Daemon101

— Jobs101

— Ingres101

— RBAC101

— Service Catalog101

— Cluster Networking101

— Network Policies 101

— Monitoring101

— Security101

— Logging101

— Helm101

Join us for this exciting one-day workshop and take your first step towards mastering Kubernetes! Don’t miss out on this opportunity to learn from experienced Kubernetes practitioners and connect with fellow beginners in the Kubernetes community. Register now to secure your spot and get ready to level up your Kubernetes skills!

Kubernetes Labs and Tutorials:

— https://github.com/collabnix/kubelabs...

— https://github.com/collabnix/kubelabs...

— https://github.com/collabnix/kubelabs...

— https://github.com/collabnix/kubelabs...

— https://github.com/collabnix/kubelabs..

. — https://github.com/collabnix/kubelabs...

Connect with us:-

• Slack: https://launchpass.com/collabnix

DevOps is a rapidly growing field that brings together the worlds of development and operations. It is a culture and set of practices that emphasizes collaboration and communication between software developers and IT professionals, while automating the process of software delivery and infrastructure changes. DevOps is transforming the way businesses approach software development and IT operations, and it has become an essential part of any modern software development organization.

In order to help foster a sense of community and collaboration in the DevOps space, I am excited to announce the creation of a new Meetup group: Collabnix Community. Our group will focus on DevOps, Cloud-native and IoT based events, bringing together experts and enthusiasts to share their knowledge and experiences.

Let’s talk about our Meetup Events…

Collabnix Community will be hosting events in collaboration with leading Meetup communities all across Bengaluru and India. Our aim is to create a vibrant and engaging community where people can come together to learn, share and network.

Collabnix Community has already established a strong presence on both Slack and Discord, with close to 10,000 members across both platforms. We believe that by extending our reach to Meetup, we can provide even more opportunities for people to connect and engage with the DevOps community.

Our events will cover a wide range of topics related to DevOps, including best practices for continuous integration and delivery, containerization, Kubernetes, serverless computing, automation, cloud infrastructure, and much more. Our speakers will be experts in their respective fields, sharing their knowledge and insights with the community.

Here’s a glimpse of our upcoming Meetup events in the next 2 months:

What benefits can you expect by joining Collabnix Community?

One of the key benefits of joining Collabnix Community is the opportunity to connect with other like-minded individuals who share a passion for DevOps. Networking is an essential part of any career, and being part of our community will provide you with opportunities to meet new people, share your experiences, and learn from others.

Another benefit of joining our community is access to exclusive content and resources. We will be sharing valuable content with our members, including webinars, whitepapers, and other resources that can help you stay up to date with the latest trends and best practices in the DevOps space.

Projects Contributed by Collabnix Community Members:

• Docker Labs — https://dockerlabs.collabnix.com
• Kube Labs — https://kubelabs.collabnix.com
• Kube Tools — https://kubetools.collabnix.com
• Docker Tools — https://dockertools.collabnix.com
• Docker Extensions — https://dockerextensions.collabnix.com
• Terraform — https://terraform.collabnix.com
• DevOps — https://devops.collabnix.com

In conclusion, if you are a DevOps enthusiast or professional looking to connect with others in the industry, I encourage you to join Collabnix Community on Meetup. Our events will be informative, engaging, and provide plenty of opportunities to network with other DevOps professionals. With our strong presence on Slack and Discord, and our growing community on Meetup, we are confident that Collabnix Community will become one of the leading DevOps communities in India.

Click Here to Join our Meetup Community

FastAPI is a modern, fast (hence the name), web framework for building APIs with Python. It is built on top of Starlette for the web server, Pydantic for data validation and serialization, and provides automatic API documentation using OpenAPI and JSON Schema.

The architecture of FastAPI is based on the following key components:

• ASGI: FastAPI is based on the ASGI (Asynchronous Server Gateway Interface) specification, which allows for high-performance asynchronous request handling.
• Starlette: FastAPI is built on top of Starlette, a lightweight ASGI framework for building high-performance async web services. Starlette provides a lot of the underlying functionality of FastAPI, such as request and response handling, routing, and middleware support.
• Pydantic: FastAPI uses Pydantic, a data validation and serialization library, to define the data models for API input and output. Pydantic provides powerful validation and serialization capabilities that make it easy to define and validate complex data structures.
• OpenAPI: FastAPI automatically generates OpenAPI documentation for your API based on your code, making it easy to document your API and to generate client code in other programming languages.
• Dependency Injection: FastAPI provides built-in dependency injection, allowing you to easily inject dependencies (such as database connections or other services) into your API functions.

The architecture of FastAPI is designed to be fast, easy to use, and highly scalable. By leveraging the power of ASGI, Starlette, Pydantic, and OpenAPI, FastAPI provides a modern, high-performance framework for building APIs with Python.

Why containerise FastAPI?

There are several reasons why you might want to containerize a FastAPI application:

• Portability: Containers provide a lightweight and portable way to package and deploy applications, making it easy to move your application from one environment to another, such as from your local development environment to a production server.
• Consistency: By containerizing your application, you can ensure that it will run the same way in any environment, without having to worry about differences in the operating system, system libraries, or other dependencies.
• Isolation: Containers provide a level of isolation between your application and the host operating system, which can help to improve security and stability.
• Scalability: Container orchestration platforms like Kubernetes or Docker Swarm make it easy to scale your application horizontally by adding or removing container instances, providing an efficient and flexible way to handle changes in demand.
• Reproducibility: By using a Dockerfile to build your container image, you can reproduce your application environment at any time, making it easier to troubleshoot issues or test new features.

Containerizing your FastAPI application can help to streamline your development and deployment processes, improve consistency and portability, and make it easier to scale and manage your application in production.

Getting Started

Pre-requisite

• Install Docker Desktop

Step 1. Create a new directory for your project and navigate into it:

mkdir fastapi-docker
cd fastapi-docker

Step 2. Create a new Python virtual environment:

python -m venv venv

Step 3. Activate the virtual environment:

source venv/bin/activate

Step 4. Install FastAPI and uvicorn:

pip install fastapi uvicorn

Step 5. Create a new file called main.py and add the following

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

@app.get("/items/{item_id}")
def read_item(item_id: int, q: str = None):
    return {"item_id": item_id, "q": q}

Step 6. Create a new file called Dockerfile and add the following code:

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9

COPY ./app /app

This Dockerfile is based on the tiangolo/uvicorn-gunicorn-fastapi image, which includes everything needed to run a FastAPI application using uvicorn and gunicorn.

The COPY command copies the contents of the app directory (which will contain our main.py file) into the /app directory in the Docker container.

Step 7. Create a new directory called app and move the main.py file into it:

mkdir app
mv main.py app/

Step 8. Build the Docker image:

docker build -t fastapi-docker .

This command will build the Docker image using the Dockerfile in the current directory and tag it with the name fastapi-docker.

Step 9. Run the Docker container:

docker run -p 8000:80 fastapi-docker

This command will start a new Docker container and map port 80 inside the container to port 8000 on the host machine.

You can now test your FastAPI application by visiting http://localhost:8000 in your web browser or making HTTP requests to it using a tool like curl.

$ curl http://192.168.0.13:8000
{"Hello":"World"}[node2] (local) root@192.168.0.12 ~
$

That’s it! You have now containerized a FastAPI application using Docker.

Using Docker Compose

If you’re looking out for Docker compose-specific sample app, head over to Docker Awesome-Compose repository

Clone the Repository

git clone https://github.com/docker/awesome-compose/
cd awesome-compose/fastapi

services:
  api:
    build: .
    container_name: fastapi-application
    environment:
      PORT: 8000
    ports:
      - '8000:8000'
    restart: "no"

Running the Containers

docker compose up -d

Listing the containers

docker compose ps
NAME                  COMMAND             SERVICE             STATUS              PORTS
fastapi-application   "/start.sh"         api                 running             80/tcp, 0.0.0.0:8000->8000/tcp

After the application starts, navigate to http://localhost:8000 in your web browser and you should see the following json response:

{
"message": "OK"
}

Stop and remove the containers

$ docker compose down

With the latest Docker Desktop 4.17 release, the Docker team introduced Docker Scout. Docker Scout is a collection of software supply chain features that appear throughout Docker user interfaces and the command line interface (CLI). These features provide detailed insights into the composition and security of container images.

How is Docker Scout different from other security tools?

There are various security tools available today in the market. While there are similarities and differences between all tools of this nature, Docker Scout stands out by offering both visibility into the dependencies called in specific layers of the images, and remediation options directly in existing developer workflows. Particularly, Advanced image analysis not only lets you analyze the vulnerabilities in your dependencies but also provides you with remediation options that help you to act quickly.

If a vulnerability is present in the base image, Docker Scout will check for any updated or patched base images and make recommendations to replace the base image. If a vulnerability is present in other layers, Docker Scout will indicate exactly where the vulnerability is introduced, and make recommendations accordingly.

Docker Scout is designed with developers in mind and integrated into Docker. With Docker Scout, spend less time searching for and fixing vulnerabilities, and more time developing your code. Docker is building Docker Scout to sit as a layer on top of the Docker ecosystem to help developers build and maintain a secure software supply chain. Right now, Docker is focussed on helping with vulnerability remediation; we think our CVE-to-package matching (using PURLs to help avoid false positives) and our SBOM-to-CVEdb matching (no need to rescan) are both nice improvements to the current Developer experience.

What problem does it solve?

1. Image Content Analysis

Docker Scout analyzes the image contents and generates a detailed report of packages and vulnerabilities that it detects. It provides you with suggestions for how you can remediate issues discovered by the image analysis. It also allows you to get base image updates and recommended tags and digests, and filter your images on vulnerability information.

2. Artifactory Integration

Docker Scout allows you to pull and view analysis for images from Docker Hub and Artifactory repositories. Integrating Docker Scout with JFrog Artifactory lets you run image analysis automatically on images in your Artifactory registries. This integration is made possible by a monitoring agent. The agent is a standalone service that analyzes images and uploads the result to Docker Scout. You can view the results using the Docker Scout web UI.

• It also allows you to get base image updates and recommended tags and digests, and filter your images on vulnerability information.

Please note that Docker Scout is an early-access product, and requires a Pro, Team, or Business subscription.

Getting Started with Docker Scout

Pre-requisites

• Docker Desktop 4.17
• Requires a Pro, Team, or Business subscription
• Enable Docker Scout on Docker Hub

Enabling Docker Scout on Docker Hub

• Login to https://hub.docker.com
• Choose the organization that contains the repository with the image(s) you want to analyse.
• Select a repo, and open the “Settings” tab.
• Under “Image insight settings”, choose “Advanced image analysis with Docker Scout.”
• View your image analysis.

Command Line Tool for Docker Scout

Using the CLI, you can analyze images and view the analysis report in text format. You can print the results directly to stdout, or export them to a file using a structured format, such as Static Analysis Results Interchange Format (SARIF).

docker scout

Usage:  docker scout COMMAND

Command line tool for Docker Scout

Commands:
  cves        Display CVEs identified in a software artifact
  version     Show Docker Scout version information

Run 'docker scout COMMAND --help' for more information on a command.

Checking the Version

docker scout version
version: v0.6.0 (go1.19.5 - darwin/arm64)
git commit: aabe2bfd192f7ac8cbfa4afea647b4dc41d3d30d

Displaying CVEs identified in a Software artifact

The docker scout cves command analyzes a software artifact for vulnerabilities. The tool analyzes the provided software artifact, and generates a vulnerability report.

Usage:  docker scout cves [OPTIONS] IMAGE|DIRECTORY|ARCHIVE

Supported Artifacts Types

The following artifact types are supported:

• Images
• OCI layout directories
• Tarball archives, as created by docker save

Images

The tool analyzes the provided software artifact and generates a vulnerability report. By default, the tool expects an image reference, such as:

• redis
• curlimages/curl:7.87.0
• mcr.microsoft.com/dotnet/runtime:7.0

If the artifact you want to analyze is an OCI directory or a tarball archive, you must use the — type flag.

Examples:

Display vulnerabilities grouped by package

docker scout cves redis
Analyzing image redis
    ✓ Pulled
    ✓ Image stored for indexing
    ✓ Indexed 144 packages
    ✗ Detected 19 vulnerable packages with a total of 56 vulnerabilities

  0C   16H    4M    1L  stdlib 1.18.2
pkg:golang/stdlib@1.18.2

    ✗ HIGH CVE-2022-30580 [Improper Control of Generation of Code ('Code Injection')]
      https://dso.docker.com/cve/CVE-2022-30580
      Affected range : >=1.18.0
                     : <1.18.3
      Fixed version  : 1.18.3
      CVSS Score     : 7.8
      CVSS Vector    : CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H
...
...
The image contains 19 packages with one or more vulnerabilities for a total of 50 vulnerabilities
  LOW      | 28
  MEDIUM   | 4
  HIGH     | 18
  CRITICAL | 0

Display vulnerabilities from a docker save tarball

$ docker save redis > redis.tar

docker scout cves --type archive redis.tar
Analyzing archive redis.tar
    ✓ Archive read
    ✓ Image stored for indexing
    ✓ Indexed 144 packages
    ✗ Detected 19 vulnerable packages with a total of 56 vulnerabilities

  0C   16H    4M    1L  stdlib 1.18.2
pkg:golang/stdlib@1.18.2

    ✗ HIGH CVE-2022-30580 [Improper Control of Generation of Code ('Code Injection')]
      https://dso.docker.com/cve/CVE-2022-30580
      Affected range : >=1.18.0
                     : <1.18.3
      Fixed version  : 1.18.3
      CVSS Score     : 7.8
      CVSS Vector    : CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H
....
...

The image contains 19 packages with one or more vulnerability for a total of 50 vulnerabilities
  LOW      | 28
  MEDIUM   | 4
  HIGH     | 18
  CRITICAL | 0

Display vulnerabilities from an OCI directory

$ skopeo copy --override-os linux docker://alpine oci:redis

skopeo copy --override-os linux docker://alpine oci:redis
Getting image source signatures
Copying blob af6eaf76a39c done
Copying config 62aab729a2 done
Writing manifest to image destination
Storing signatures

docker scout cves --type oci-dir redis 
Analyzing OCI directory redis
    ✓ OCI directory read
    ✓ Image stored for indexing
    ✓ Indexed 19 packages
    ✓ No vulnerable package detected

Export vulnerabilities to a SARIF JSON file

$ docker scout cves --format sarif --output redis.sarif.json redi

References

• Docker Scout Announcement Blog
• https://dso.docker.com/scout-intro
• Scout Docs