Skip to content

Update build documentation#3

Merged
zwass merged 5 commits into
fleetdm:masterfrom
noahtalerman:build-docs
Nov 3, 2020
Merged

Update build documentation#3
zwass merged 5 commits into
fleetdm:masterfrom
noahtalerman:build-docs

Conversation

@noahtalerman
Copy link
Copy Markdown
Member

No description provided.

@zwass zwass added the :improve documentation Involves writing improvements or additions to documentation label Nov 3, 2020
@zwass zwass changed the title Build docs Update build documentation Nov 3, 2020
@zwass zwass merged commit be28d8e into fleetdm:master Nov 3, 2020
getvictor added a commit that referenced this pull request Feb 22, 2024
@dherder dherder mentioned this pull request May 30, 2024
kc9wwh added a commit that referenced this pull request Jul 24, 2025
- Noted that Fleet generates a new encryption key and the original one does still remain under step #3 after a customer noted inconsistencies in our docs. 
- The statement was taken from our enforce-disk-encryption article.
noahtalerman pushed a commit that referenced this pull request Jul 29, 2025
- Noted that Fleet generates a new encryption key and the original one
does still remain under step #3 after a customer noted inconsistencies
in our docs.
- The statement was taken from our enforce-disk-encryption article.
jacobshandling added a commit that referenced this pull request Sep 15, 2025
…mium (#3… (#33017)

See #33006

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
nulmete added a commit that referenced this pull request Jan 23, 2026
…tu 20.04 during fleet-desktop update (#38648)

<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #35413 

Applied the same fix as in
https://github.com/fleetdm/fleet/pull/29186/files:
- Added musl-tools to the container image. This provides musl-gcc, a
compiler that links against musl libc instead of glibc. We use it for
static linking as explained below.
- Added static linking flags (CGO_ENABLED=1 CC=musl-gcc -linkmode
external -extldflags "-static"). This produces a self-contained binary
with all C library code embedded, eliminating runtime dependencies on
the host system's glibc version.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

Steps:

- Started TUF server for the first time. Generated a Linux x86_64 image.

```
SYSTEMS="linux" \                                      
DEB_FLEET_URL=https://nicofleet.ngrok.io \
DEB_TUF_URL=http://nicotuf.ngrok.io \
GENERATE_DEB=1 \
ENROLL_SECRET=tm2CHBEF1I5BVuM1+4hzRRtpC5ZYV8vb \
FLEET_DESKTOP=1 \
DEBUG=1 \
./tools/tuf/test/main.sh
```

- Installed `fleet-osquery_26.1.46030_amd64.deb` generated by the
previous command on a Kubuntu 20.04 x86_64 VM.
- Ran `sudo journalctl -u orbit` to debug logs and verified that the
GLIBC incompatibility error was raised:

```
tammi 22 15:36:53 nicolas-Standard-PC-i440FX-PIIX-1996 sudo[125623]: pam_unix(sudo:session): session opened for user nicolas by (uid=0)

tammi 22 15:36:53 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125624]: /opt/orbit/bin/desktop/linux/stable/fleet-desktop/fleet-desktop: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /opt/orbit/bin/desktop/linux/stable/fleet-desktop/fleet-desktop)

tammi 22 15:36:53 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125624]: /opt/orbit/bin/desktop/linux/stable/fleet-desktop/fleet-desktop: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /opt/orbit/bin/desktop/linux/stable/fleet-desktop/fleet-desktop)

tammi 22 15:36:53 nicolas-Standard-PC-i440FX-PIIX-1996 sudo[125623]: pam_unix(sudo:session): session closed for user nicolas

tammi 22 15:37:08 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125199]: 2026-01-22T15:37:08+02:00 INF killing any pre-existing fleet-desktop instances

tammi 22 15:37:09 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125199]: 2026-01-22T15:37:09+02:00 INF attempting to get user session type and display id=1000 user=nicolas

tammi 22 15:37:09 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125199]: 2026-01-22T15:37:09+02:00 ERR failed to get X11 display, using default :0 error="display not found on who output"
```

- Built a new version of the agent after applying the fixes on this PR
and pushed it to the TUF server:

```
# 1. Hardcode orbit to a higher version
export ORBIT_VERSION=26.1.46099

#2. Generate new package
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-linux

#3. Update to TUF server
./tools/tuf/test/push_target.sh linux desktop desktop.tar.gz $ORBIT_VERSION
```

<img width="396" height="179" alt="Screenshot 2026-01-22 at 5 18 25 PM"
src="https://github.com/user-attachments/assets/56182580-1d54-4945-af03-98762e7795e9"
/>


- In the VM, verified that an update for fleet desktop was detected by
running `sudo journalctl -u orbit -g "update detected"`:

```
tammi 22 20:33:32 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[4114]: 2026-01-22T20:33:32+02:00 INF update detected target=desktop
```

- Verified the new version is shown both on the desktop icon and the
Fleet UI:

<img width="1373" height="248" alt="Screenshot 2026-01-22 at 5 22 17 PM"
src="https://github.com/user-attachments/assets/9fac73ac-a3a6-49ed-82c7-261abec43798"
/>

<img width="1420" height="496" alt="Screenshot 2026-01-22 at 5 22 46 PM"
src="https://github.com/user-attachments/assets/55321108-9233-4799-b5e5-0713172c0138"
/>


## fleetd/orbit/Fleet Desktop

- [ ] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
TsekNet added a commit to TsekNet/fleet that referenced this pull request Mar 22, 2026
- Extract nested block to tryReuseExistingInstaller helper (fleetdm#5)
- Add 30s timeout to HEAD request client (fleetdm#2)
- Validate URL scheme (http/https only) for SSRF defense (fleetdm#1)
- Weak ETag comparison: strip W/ prefix per RFC 7232 (fleetdm#3)
- Validate ETag/Last-Modified format before storing (fleetdm#6)
- Move HTTPETag/HTTPLastModified into fillSoftwareInstallerPayloadFromExisting (fleetdm#9)
- Remove duplicate store-existence check (fleetdm#7)
- Add ORDER BY si.id DESC to LIMIT 1 query (fleetdm#8)
- Use ds.writer (primary) for GetInstallerByTeamAndURL (fleetdm#13)
- Rename checkURLChanged to hasURLContentChanged (fleetdm#11)
- Rename urlContentUnchanged to canSkipDownload (fleetdm#16)
- Add nil guard in mock to prevent panic in existing tests
- Fix schema.sql collation to match migration output
- Fix lint: use svc.logger instead of bare slog.Warn
- Add tests: weak ETag, both headers precedence, 403/500 status,
  non-HTTP scheme, normalizeETag, validETag (fleetdm#3,12,14,17)
- Document redirect limitation (fleetdm#15)
getvictor added a commit that referenced this pull request Apr 6, 2026
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #42836 

This is another hot path optimization.

## Before

When a host submits policy results via `SubmitDistributedQueryResults`,
the system needed to determine which policies "flipped" (changed from
passing to failing or vice versa). Each consumer computed this
independently:

```
SubmitDistributedQueryResults(policyResults)
  |
  +-- processScriptsForNewlyFailingPolicies
  |     filter to failing policies with scripts
  |     BUILD SUBSET of results
  |     CALL FlippingPoliciesForHost(subset)          <-- DB query #1
  |     convert result to set, filter, queue scripts
  |
  +-- processSoftwareForNewlyFailingPolicies
  |     filter to failing policies with installers
  |     BUILD SUBSET of results
  |     CALL FlippingPoliciesForHost(subset)          <-- DB query #2
  |     convert result to set, filter, queue installs
  |
  +-- processVPPForNewlyFailingPolicies
  |     filter to failing policies with VPP apps
  |     BUILD SUBSET of results
  |     CALL FlippingPoliciesForHost(subset)          <-- DB query #3
  |     convert result to set, filter, queue VPP
  |
  +-- webhook filtering
  |     filter to webhook-enabled policies
  |     CALL FlippingPoliciesForHost(subset)          <-- DB query #4
  |     register flipped policies in Redis
  |
  +-- RecordPolicyQueryExecutions
        CALL FlippingPoliciesForHost(all results)     <-- DB query #5
        reset attempt counters for newly passing
        INSERT/UPDATE policy_membership
```

Each `FlippingPoliciesForHost` call runs `SELECT policy_id, passes FROM
policy_membership WHERE host_id = ? AND policy_id IN (?)`. All 5 queries
hit the same table for the same host before `policy_membership` is
updated, so they all see identical state.

Each consumer also built intermediate maps to narrow down to its subset
before calling `FlippingPoliciesForHost`, then converted the result into
yet another set for filtering. This meant 3-4 temporary maps per
consumer.

## After

```
SubmitDistributedQueryResults(policyResults)
  |
  CALL FlippingPoliciesForHost(all results)           <-- single DB query
  build newFailingSet, normalize newPassing
  |
  +-- processScriptsForNewlyFailingPolicies
  |     filter to failing policies with scripts
  |     CHECK newFailingSet (in-memory map lookup)
  |     queue scripts
  |
  +-- processSoftwareForNewlyFailingPolicies
  |     filter to failing policies with installers
  |     CHECK newFailingSet (in-memory map lookup)
  |     queue installs
  |
  +-- processVPPForNewlyFailingPolicies
  |     filter to failing policies with VPP apps
  |     CHECK newFailingSet (in-memory map lookup)
  |     queue VPP
  |
  +-- webhook filtering
  |     filter to webhook-enabled policies
  |     FILTER newFailing/newPassing by policy IDs (in-memory)
  |     register flipped policies in Redis
  |
  +-- RecordPolicyQueryExecutions
        USE pre-computed newPassing (skip DB query)
        reset attempt counters for newly passing
        INSERT/UPDATE policy_membership
```

The intermediate subset maps and per-consumer set conversions are
removed. Each process function goes directly from "policies with
associated automation" to "is this policy in newFailingSet?" in a single
map lookup.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Performance Improvements**
* Reduced redundant database queries during policy result submissions by
computing flipping policies once per host check-in instead of multiple
times.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
akuthiala pushed a commit that referenced this pull request May 18, 2026
@claude claude Bot mentioned this pull request Jun 2, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:improve documentation Involves writing improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants