I want to use a TODO application that takes into account where I need to be to complete a task. I don’t “buy milk” while at home nor do I “cut the grass” when I’m at work. I want my TODO application to be smart and only show me tasks that make sense for where I am or where I’m going.

TodoMVC

If you are unfamiliar with TodoMVC, it was started by Addy Osmani and Sindre Sorhus sometime around 2012 with the help of many contributors over the years as “a project which offers the same Todo application implemented in most of the popular JavaScript MV* frameworks of today — think of it as speed dating for frameworks.” (1)

With the TodoMVC React implementation as a starting point we’re going to start adding location features.

• [ ] Add latitude and longitude with a TODO React component
• [ ] Display TODOs with a marker on a Map component

Additionally, there is a little bit of housekeeping to update the sample with the latest ES6 and React functionality.

// TODO: Run “Hello World”

There are at least two paths for getting a “Hello World” started quickly.

If you are more of a deconstructionist that wants to start with a working app and take it apart to understand how it works:

git clone https://github.com/tastejs/todomvc.git
cd todomvc
python -m SimpleHTTPServer

You should be able to view http://localhost:8000 and navigate to the React example found in examples/react. If you are more of a constructionist and want to start with a blank slate and build up by adding piece by piece:

npx create-react-app my-todo-app
cd my-todo-app
npm start

I prefer this approach, so when create-react-app finishes you should be able to view http://localhost:3000 with live reloading and a basic Hello World React App.

// TODO: Add A Component

A React component is a structure for creating independent, reusable pieces of a user interface. The component accepts properties as input and returns a React element that can be rendered.

TodoMVC originated with a goal of implementing an architecture by which the **M**odel, **V**iew, and **C**ontroller were independent. React aims to not artificially separate technologies such as JavaScript from HTML/CSS but rather separate concerns with loosely coupled components that contain both. The Reactjs.org [Main Concepts](https://reactjs.org/docs/components-and-props.html) goes into more detail on this design approach.

Review the following component source. It defines a simple text input box that defaults to the current location. There are inline comments to help describe some of the logic.

import React, { Component } from 'react';

// This class definition is a React.Component so that we
// can use it in multiple places for the app.

class Location extends Component {

// The constructor takes properties defined as element attributes
  // defined in JSX along with an initial default value for state.

constructor(props) {
    super(props);
    this.state = {
      value: '0,0', // Null Island
      error: null,
    }
  }

// When the component is rendered to the DOM for the first time
  // such as at page load we call the Geolocation API to determine
  // a latitude and longitude for the browser

componentDidMount() {
    if (navigator.geolocation) {
      navigator.geolocation.getCurrentPosition(
        (position) => {
          this.setState({
            value: position.coords.latitude + ',' + position.coords.longitude,
            error: null,
          });
        },
        (error) => this.setState(
          {error: error.message}
        )
      );
    }
  }

// Respond to user input with event callback
  
changeLocation(evt) {
    this.setState({
        value: evt.target.value,
      }
    )
  }

// The JSX definition for how to render this component on the page.  
  // In this case, it's a simple input field for new todo items.

render() {
    return (
            <input
              className="new-todo"
              value={ this.state.value }
              onChange={ evt => this.changeLocation(evt) }
              />
    );
  }
}

// TODO: Add a Map

For the map, we’re going to use the Map Image API which provides a quick and easy way to fetch a static image. The parameters take some getting used to but let me unpack the basics here:

• w=600 specifies the width of the image tile requested and h=300 the height
• z=10 specifies the zoom level
• t=5 specifies the rendering scheme so you can choose from various styles of map tiles including satellite, transit, etc.
• poitxs=1 , poitxc=black , and poitfc=yellow specifies the point of interest size, background color, and foreground color
• app_idd=... and app_code=... are found in the here developer projects section and needed for working with any HERE APIs.

Each of these parameters could be stored as either props or state on the component to provide rich customization options to users. The last parameter we want to send to the Map Image API is the poi= for a comma separated list of latitude and longitude for any markers we want to place.

In the case of my TODO app, I’ve added two tasks with locations:

• “Return Library Book” at 37.86836, -122.26859
• “Pickup Badge for TechCrunch” at 37.784117,-122.401386

The poi query we’d want to make for these todo items would look like poi=37.86836,-122.26859,37.784117,-122.401386.

Here is an example of the Map component that will re-render with each setState() call when adding points of interest:

class Map extends Component {

// For conciseness simply included all parameters in the querystring directly

constructor(props) {
    super(props);
    this.state = {
      url: 'https://image.maps.api.here.com/mia/1.6/mapview?w=600&h=300&z=10&t=5&poitxs=16&poitxc=black&poifc=yellow',
      points: [],
    }
  }

// Helper function to format list of points

getPOIList() {
    if (this.state.points.length > 0) {
      let param = '&poi=';
      for (var poi in this.state.points) {
        param += poi.latitude + ',' + poi.longitude;
      }
      return param;
    }

return '';
  }

// Render method builds the URL dynamically to fetch the image from the
  // HERE Map Image API

render() {
    return (
      <img
        src={ this.state.url
          + '&app_id=' + this.props.app_id
          + '&app_code=' + this.props.app_code
          + this.getPOIList()
          }
        alt="Todo Map"/>
    );
  }
}

Once defined, you can reuse this component in any other component such as the root app component.

<Map app_id=”PUT_APP_ID_HERE” app_code=”PUT_APP_CODE_HERE” />

// TODO: Wrap It Up

This first post introduced TodoMVC with a couple of very basic React components that leverage the HERE Map Image API. In upcoming React posts I’ll dive deeper into this example to respond to state changes with event handling for geocoding lookups, map clustering for TODOs at the nearby locations with conditional rendering, Matrix Routing to get between TODO locations efficiently, Place Search to find related TODO items, and a few other project integrations to polish up the final todo app.

I suspect it is not uncommon for somebody technically minded to volunteer to do something only to find themselves realizing, “ok, well, now I need to figure out how to do that” simply because they were looking for a challenge. I decided to start recording video from my son’s ice hockey games as the team videographer. I have no prior experience doing any video editing or filming of anything let alone sports, but professionally I do know how to write software even if I’m an amateur when it comes to media.

What I have observed is that parents like to be able to see their son or daughter score a goal or make a key save. Coaches like to review video and find teachable moments and lessons on teamwork. Individual players want to perform at higher and higher levels on their way to their dreams of the NHL so want to review video just like the pros to refine their own techniques and results.

My goal is not to become a professional video editor or sports analyst. I want to help the parents, coaches, and players get what they want but without spending all of my free time doing it. I want to automate, hack and take short-cuts, derive insights and still come out with something useful and professional looking with as little effort as is necessary. Larry Wall, creator of the Perl programming language, defined the Three Virtues of a great programmer which includes laziness, a quality that makes you go through great lengths to not do work later by spending even more energy up front writing programs to save labor in the long run.

There are challenges and solutions which I’ll be releasing over the course of the 2019–2020 PeeWee (12 and under) season. Broadly these lessons fall into the categories of:

• Capture — the decisions for equipment needed to capture raw audio and video from multiple locations
• Editing — telling the story of the event when not everything captured is useful or re-assembled from various perspectives
• Storage — planning for storing and archiving media
• Analysis — identifying what happened and when through a combination of video and audio analysis combined with scoresheets
• Delivery — how to get the results efficiently to the audience and stakeholders

While I’m part of an ice hockey family, there are similar communities that can benefit from similar approaches with football, soccer, baseball, lacrosse, etc.

The pre-season has wrapped up so now it’s time to put it all in practice. The questions I’ll be answering include:

• What equipment should I use to capture ice hockey footage?
• What are the ideal settings in terms of quality and frames per second?
• How to get the data from multiple camera sources loaded?
• How to manage local or cloud media file storage?
• Which settings and codecs can I use to transcode for efficient use of space and quality?
• How to synchronize multiple video feeds in a timeline?
• What can I do to improve the audio of the media to sound more like sports broadcasts?
• How to cut between cameras depending on where the action in the scene is?
• How to identify scenes within the game such as goals, penalties, or stoppages in play?
• Which editing software should I use? How to use iMovie, DaVinci, Avid, Adobe, or something else?
• How to pan / zoom cameras remotely?
• How to do privacy blurring and remove identifiable information of a minor or somebody that did not agree to audio or video?
• Is it possible to automate the insertion of titles, transitions, and backgrounds?
• How to identify events like passes or shots and the players within the frame?
• How to use streaming or other broadcast media platforms like YouTube, Vimeo, Twitch, and Mixer to share with fans and family that can’t make it to the game live?
• What are the career paths and resources for somebody that really does want to become a videographer?
• How to insert slow motion replays for key moments?
• Are there tricks to make animated GIFs or highlight videos with slices of the game such as following a specific player?
• Can I just buy a solution or is it worth building this on my own?

It’s a long season, so to find these answers, I’ll post updates on Twitter where you can find me as @jaysondelancey. It’s an exploration with a mix of the technical and creative aspects while also bringing in some experts along the way to share tips & tricks of those that do this professionally as a career.

A Showcase Project

DevRel builds interesting projects, prototypes, and demonstrations by using the company’s products and APIs to understand what an external developer’s experience is. (link)

As a Developer Evangelist with General Electric, I was often on the lookout for what I referred to as a showcase or hero project. Writing a tutorial, how-to, or other short demonstration can be done in a few days and can quickly validate that a service or API does what it says it does. Those are important, but to get a deeper understanding of what a developer’s experience is can come from using the product to build and support something of a larger scope that more closely resembles a production application or purpose.

In 2017, I learned about a collaboration that was beginning as part of a marketing campaign. A team was being assembled to go on an expedition to Nicaragua and install industrial IoT sensors in a volcano. It was a purpose-based mission to investigate the potential for outcomes like early-warning systems, preventative maintenance, and just a better understanding of our world. I’m not going to rehash it all here so to learn more about the story, project, and people involved — enjoy the website for an engaging scientific exploration (https://ge.com/digitalvolcano) but come back, I have more to tell about DevRel below.

Inspiring Others

It sounded like a fascinating project when I heard the pitch so immediately asked how I could help. Ultimately, GE was not looking for any commercial viability in profiting from volcanoes, so this was an ideal project for DevRel to contribute software engineering resources to help develop. The data and source-code would not require any intellectual property protection so could be freely shared and talked about in public.

My involvement grew over time from initially just providing technical support to taking on an analytics dashboard. As an evangelist I didn’t have an engineering team to leverage, but it was an interesting enough project that I was able to persuade and recruit a half-dozen engineers from across the organization that were excited enough about the concept to contribute their personal time and energy to building the dashboard as a side-project of their regular jobs.

SDKs and Reference Apps

DevRel develops tools and SDKs that help enable developers to onboard and have success faster and with less pain. (link)

Once the industrial sensors were deployed from the expedition we were able to get a stream of data coming from the sensors to analyze. We wanted to store, query, and analyze this data so we chose Python as our development language to build a dashboard. One of our first observations was that there were no resources to help a Python developer get started. Every developer would benefit from some tools so we added that to our list of deliverables:

• Develop an SDK that can be downloaded and used in the client language. While postman and curl examples that were published by engineering and documentation teams were fine, they are a least common denominator. Every developer needs to write some client code to build an application, so why not make it easier for everybody by providing it directly so that authentication, orchestration, and error-handling were handled.
• Develop a Reference Application that can be downloaded and used to deploy an application quickly. Every developer is capable of standing up an app, but is that the best use of their time in evaluating a new product? If somebody can get a prototype up and running in an hour rather than a week — that gives a much better impression of the platform.

We felt the pain ourselves, built the resources, and made them available on GitHub.

Getting Started Guides

DevRel creates technical documentation such as formal getting started guides and tutorials as well as informal blog posts. (link)

Well written code is great but not everybody can learn from reading code alone. Often, the why and context must be explained. In DevRel, we want to help all developers with a variety of skill levels, so it’s important to explain things for both somebody who’s a beginner and for advanced cookbook recipes for solving specific technical challenges. The SDK and reference app for the volcano project were delivered with complete documentation to help others learn.

Getting the Word Out

DevRel attends, speaks, and sponsors developer events like meetups, conferences, and hackathons delivering content through a variety of modalities like presentations, webinars, live coding, recorded videos, podcasts, workshops, etc. (link)

It is considered taboo to sell a product with Developer Relations. Ultimately, an organization exists to make a profit though so if the product is priced correctly and the value exceeds its cost then anybody whether developer or not would consider buying. Therefore, DevRel does exist to “sell” but achieves this result indirectly.

• DevRel focuses on a long-term relationships rather than short-term transactions
• DevRel focuses on providing something of immediate value for free whether somebody is buying or not

That value frequently comes from the transfer of knowledge and access. To accomplish this, content is developed and delivered to wherever an audience exists that might benefit. The volcano project resulted in a lot of learning that can be shared with no strings attached.

• what our services were good for
• what our services were not good for
• what we learned from using those services in combination with other related technologies
• what we learned about those related technologies
• the tools we created that might make it easier for others trying to use any of our services or related technologies

We had a new SDK and Reference App that we thought would be useful to others. Some developers might stumble upon them if they looked, but it was time to get the word out more proactively. We believed others might be interested in engaging with us long-term and could benefit from what we learned.

What did that campaign look like?

We released blog posts on multiple sites telling the story in different ways.

We engaged Media Relations and journalists covered the story in CNET, Live Science, Observer, etc..

We posted to aggregators like HackerNews, Reddit, and Product Hunt.

We created videos, podcasts, and webinars so that others could listen at the time, place, and format they preferred.

We wanted to answer questions both live at events and through online support forums.

DevRel should avoid the bad rep that comes from a “sales pitch” by providing value through knowledge. To accomplish that requires a lot of engagement both online and in-person. It’s a job where functions like engineering, product, marketing, etc. should participate but as the commitment becomes a full-time job that’s why Developer Relations exists as an organization.

Summary

What does DevRel do? Installing sensors in a volcano is not your typical project. Most projects don’t need to be at this scope or scale to be effective (was this the most expensive SDK launch ever?) but it does illustrate a few best practices for doing developer relations:

• build something with your product, the experience first-hand is a good learning opportunity
• provide tools like SDKs, reference apps, etc. that would have helped you because they’ll also help others
• document the journey, the mistakes and learning you had along the way can help others save time
• spread the word in as many places and ways as possible

This was a project that got a high level of return for the engineering investment that went into it. Without DevRel this project would not have happened or had as much of an impact in helping other developers.

What is DevRel?

Here’s my take.

Developer Relations (or DevRel) is the part of an organization that focuses on making sure a product is ready for adoption when the intended user is a software developer. A technical product is different because the audience for consuming it:

• doesn’t typically respond to traditional marketing tactics
• doesn’t typically respond to traditional sales
• is well informed and often capable of solving the problem themselves

A product that is an API available on the cloud is a prime example, developers can either recommend or block the buying decision. Therefore, DevRel exists as a cross-functional team that aspires to earn that developer’s recommendation.

This can be done by ensuring a great developer experience, evangelizing the product to increase awareness, building a community within larger software ecosystems, advocating for changes in the product that help developers with adoption, and providing the support needed for developers to be successful building their own project by using yours.

Many large technology companies have DevRel teams and programs including AWS, Dolby, Google, Intel, IBM, Microsoft, Salesforce, Samsung, Twilio, etc.

What Does DevRel Do?

There are a number of activities that can be grouped into the domain of Developer Relations. Depending on the stage of a product in its journey from development to general availability in the market will drive where DevRel’s priorities are.

• Attend, speak, and sponsor developer events such as meetups, conferences, and hackathons
• Build interesting projects, prototypes, and demonstrations by using the products and APIs first
• Create technical documentation both formal getting started guides or tutorials as well as informal blog posts
• Deliver content through a variety of modalities — webinars, live coding sessions, recorded videos, podcast interviews, in-person workshops or presentations, etc.
• Work with partnerships & alliances for joint go to market collaboration opportunities with complementary services needed in building something
• Grow communities online & offline through newsletters, VIP programs, etc.
• Provide technical support across channels that are convenient for developers including in-person events, online forums, stack overflow, quora, social media, etc.
• Develop tools and SDKs that help enable developers to onboard and have success faster and with less pain

Digital Volcano Case Study — look at my other story on a developer campaign that demonstrated some of these activities in action.

To accomplish these activities DevRel must work in close collaboration with participation and interlock from many other departments across the organization such as Product Management, Engineering, Marketing, Communications, Public Relations, Legal, and Customer Support. This is part of what makes placing DevRel within an organization a common challenge. The whole organization must contribute to the mission, DevRel just focuses on it full-time across functions.

What Skills Does Somebody in DevRel Have?

The #1 most important skill for Developer Relations to have in their skillstack is empathy.

I was once told that at Google, the Developer Advocate is viewed as customer zero — an early stop in the release process to feel what the end developer will experience when trying to use an API in a project. This is not the same objective as Quality Assurance (QA) but more like user acceptance testing to get “Hello World” running efficiently within a developer’s preferred toolset. This means making sure the service is easy to learn, can be used in different environments, and can be used in combination with other products offered by the same company.

A DevRel organization requires a cross section of skills like:

• Software Development —a background in solutions architecture, polygot full-stack development use of web APIs, etc.
• Communications — such as technical writing and public speaking, cross-functional interpersonal relationships
• User Experience (UX) — such as User Centered Design, building developer productivity tools, etc.
• Business — such as business development, partnerships & alliances, public relations, etc.

Depending on any individual’s specific strengths might prepare them for one of a number of typical developer relations roles (this is not the structure of a team, just examples of titles you might see):

• Developer Evangelist / Technical Evangelist — definitions vary, but frequently associated with representing the company to the community in order to sell the dream of what a product can do
• Developer Advocate — often puts greater emphasis on representing the community to the company, expressing the feedback gathered from developers in the community to help shape the product direction while giving support
• Community Manager / Events Manager — networker and facilitator for looking after the community, sharing information, making connections, or planning events
• Developer Experience / Developer Programs Engineer — often works within the company by focusing on usability and engineering of developer tooling, hygiene around the APIs, building SDKs, tutorials and portals
• Content Manager / Technical Writer — skilled at communicating clearly and effectively to audiences
• Developer Marketing / Growth Hacking — succinctly crafting the message around what a product does so that it resonates with the developer audience trying to find it by cultivating social media, advertisements, and related tactics.

Summary

I have been practicing Developer Relations for a number of years with direct visibility across multiple organizations so felt I could finally synthesize my perspective on what DevRel is for its impact on a business and industry. Many other sources influenced my point of view, so I’d call your attention to these other resources for additional perspectives:

• Christian Heilmann’s Developer Evangelist Handbook
• Matthew Revell’s DevRelCon and devrel.net
• Phil Leggetter’s Defining Developer Relations
• Brandon West’s The 3 C’s of Developer Relations
• Mary Thengvall’s The Business Value of Developer Relations and What Is Developer Relations (And Why Should You Care?)
• Ashley McNamara’s What is Developer Advocacy?
• Robert Nyman’s What Developer Relations Is and How to Do It
• Reto Meier’s The Core Competencies of Developer Relations
• …and many more

Community — Makers

I align myself closely with the maker community and had the fun task of building out a makerspace at GE Digital in San Ramon. With a small budget and the help of Rik Dryfoos we furnished the makerspace with 3D Printers, tools, electronic components, and a number of demos to help explain Industrial IoT concepts to corporate guests who visited the GE Digital Foundry. We also developed some workshops and small group training while collaborating with other maker communities within GE at other offices such as Detroit, New Orleans, and then Crotonville.

Community — Open Source

Another community that demonstrates the spirit of a rising tide lifts all boats is the open-source organizations within a company. Upon joining HERE Technologies, I was delighted to find the engineering teams actively engaged and doing some amazing work. I took a bit of a deep dive into a few of the projects that have been generously shared with the world in order to shine the spotlight on the engineers making it happen.

Visualizing Large Scale 3D Terrain with Open Source Tools - HERE Developer

J12Y on Twitter

Learn from Victor Lu's blog post on the open-sourced Python package from @heredev called the Point Processing Tool Kit (PPTK) for visualizing millions of 3D point clouds such as GPS trajectories & LIDAR point clouds. A nice gift before the holidays. https://t.co/KnQx57f1Tl

Events

In the second half of 2018 I found myself on the road quite a bit. Including internal meetings, hackathons, and conferences where I visited Seattle, Los Angeles, Chicago, New York, Detroit, Kansas City, Austin, and Berlin.

Events — Speaking

To share things I’ve learned along the way that might save somebody some time I had a few speaking sessions:

• Silicon Valley Code Camp — The Location of Things from A to XYZ
• Developer Week — Making Maps with XYZ
• AWS re:Invent — The Importance of Location for IoT
• Webinar — Mapping Seismic Activity, an Intro to HERE XYZ

The first webinar is just the start and we are ready to announce many more coming in January.

Events — Engaging

Events are a great place to share information through speaking but even more than that it is a great way to learn about what others are building and ask questions. There were a handful of events where HERE Technologies sponsored hackathons or booths to meet with folks in the communities.

• Hack Midwest
• TechCrunch
• CalHacks
• DroidCon NYC
• State of the Maps US

J12Y on Twitter

Congrats to team "Collector" for creating a fun geocaching application using @reactjs and @heredev at @CalHacks this weekend. Great to meet the team behind it too. #CalHacks5 #TeamPreston

Another great aspect of going to events is it is also a good chance to catch up with former colleagues when everybody has moved on to new roles and companies.

Content

A key responsibility of my role is to advocate on behalf of the community internally to product teams. One way I do this is by building projects myself to stumble over any pain points before others do and channel that back as feedback to engineering or teach others some of the things I learned along the way in written tutorials.

Content — React

I haven’t been a front-end web developer for several years but I wanted to get back up to speed with one of the more popular frameworks — React. There had been a number of questions from the developer community about how to be successful with web components and location so I spent some time writing blog posts to describe some of the projects I built after learning more about how to use the library better myself.

• HERE Matrix Routing with ReactJS to Optimize Your Black Friday
• Use HERE Interactive Maps with ReactJS to Pick a Theme
• Street Address Form Validation with ReactJS and HERE Geocoder Autocomplete
• A Location Based TODO App

J12Y on Twitter

How to Use HERE Interactive Maps with React JS, a tutorial and demo of building a map theme / style picker @heredev #ReactJS #JavaScript https://t.co/hvTscWifx3

I have a few more projects in progress with ReactJS so expect a few more over the next few months.

Content-Google Alternatives

Another theme I focused on was to provide some guidance for developers and engineering teams feeling the impact of Google’s decision to raise prices. Google has been successful with their consumer application that many developers didn’t even realize there were alternatives such as HERE Technologies, myself included. Once you get a chance to explore the HERE services you often can build the same applications. I wrote a few posts to help highlight a few difference to try and make those migrations easier:

• How-to Migrate KML to HERE Maps
• How-to Migrate from Google Routes to HERE Routing
• How-to Migrate from Google Places to HERE Geocoder
• How-to Migrate from Google Maps to HERE Maps

We also put together a freemium SwitchToHERE site that helps learn the basics like these and covers a few more done with the help of some of my teammates.

Content-Developer Blog

I’ve spent a great deal of time to help shape the voice and technical depth of the HERE Developer Blog by authoring posts myself, working with teammates, and mentoring other writers to help them tell their story. Here’s a sample:

• Making Maps with HERE XYZ
• Building an App for Routing to Nearby Water Fountains by Boris Guenebaut
• Turn Text Into Maps with Python NLTK
• Safely Route College Students Using the HERE iOS SDK by Brandon David
• Run Circles Around GeoJSON, Beginner to Advanced

Know Your Location — Voiding My Firefox Warranty

What’s Next

I’ve been gathering and preparing a lot more events and content for 2019 so if you like Python, Open-Source, Robotics, Autonomous Driving, and Cloud Services stay tuned for more next year.

Thanks, Happy Holidays and Happy New Year.

Key Topics:

• Webinar Frequency — how often should you do a webinar, how often should you repeat content previously presented in a webinar.
• Live vs. Pre-recorded
• Software
• Scheduling

WEBINAR FREQUENCY

I’m comparing two options for a webinar frequency decision:

One and Done Strategy — Run webinar on a topic once and then make the recording available to anybody that registers after the originally scheduled event.

Amortization Strategy — Run webinar on a topic multiple times and still make the recording available, but repeat the live session again periodically (each quarter) to offer opportunity for Q&A and minor changes / updates.

EFFORT

There is effort involved in producing new content and promoting it in a webinar. A few rough estimates of the amount of time involved with each activity:

• Create new content (2–3 weeks)
• Marketing for an upcoming webinar (1–2 days)
• Deliver content at the scheduled time (1–2 hours)

There is inevitably some follow-up questions from participants as well that take time which is expected. There doesn’t seem to be an added cost of hosting more than one webinar (fixed monthly cost) and I’m assuming no advertising expenses relying only on organic leads.

RETURN

If the content being delivered has value to the audience then we should see registrations. I’d hazard a few assumptions:

• Some cannot attend due to a scheduling conflict no matter when you schedule it
• Some cannot attend because they were unaware it was happening until after the fact
• Human behavior is to commit to spending time in a scheduled webinar but less likely to watching a recording of one on-demand that can be viewed anytime
• Some will find significant value in the opportunity to participate and ask the host questions which doesn’t happen with recorded session
• Incremental cost of repeating a session is smaller than the effort of creating it the first time or creating new content for a separate webinar
• One can more successfully promote future upcoming webinars than recordings of past webinars — through newsletters, social media, web sites, etc. even if we’ve done the topic of that webinar in the past
• If the webinar was valuable, those who attended the first time will send news of a second occurrence to colleagues

SUMMARY

My hypothesis is that the Amortization Strategy is the better approach where better is measured by the number of registered participants who get value from the webinar. I’d welcome any evidence based perspectives that might change my mind and save me from a long period of learning from mistakes.

Using some fuzzy napkin math to estimate a model of attendance over the course of 12 months.

• Attendance (as with meetups) would be 1/3 of registrations though some that missed will probably watch the recording later
• For n attendees, there is a factor for each subsequent event of new registrations. I’d expect the second instance to be better attended than the first if announced at the same time for instance. I’m using 1.0, 1.25, 0.75, and 0.5 as factors against n if running same content each quarter.

With that model in mind, the Amortization Strategy would yield a 352% increase in registrations per original content produced and 163% more over the course of the year even accounting for the opportunity cost of fewer new original presentations.

Originally published at developer.here.com so check out the original too.

What is in your Top 5 travel destinations? I asked my 8 year old son this recently and the response surprised me. The answer was very specific and unfamiliar.

OK, I know Venice but I’m a software engineer in Northern California and haven’t studied geography in quite some time. I have to admit I have no idea where some of those places are.

Maybe you have found yourself reading a travel article that beautifully describes a glamorous locale, places to stay, visit, and where to eat. These articles can be swell, but a simple map can go a long way to add the context of place that some expertly crafted prose alone cannot do. If the author or publisher didn’t include a map for you and you don’t have an 8-yr old Geography savant in your house, Python can help.

Solution

To solve this problem we will stand up a Python Flask server that exposes a few APIs to

1. Download a given URL and parse the HTML with BeautifulSoup.
2. Extract locations from the text based on some clues with the Natural Language Toolkit (NLTK).
3. Geocode the location to determine a latitude and longitude with the HERE Geocoder API.
4. Place markers on a map to identify the recognized places with the HERE Map Image API.

Server Setup

For this section I make an assumption you are running an environment like OSX or Linux. If you are running Windows you will need to adjust some of the commands a bit.

Configuration

With the Twelve-Factor App the case is made that a best practice is to store config in the environment. I agree and like to store my API credentials in variables APP_ID_HERE and APP_CODE_HERE found in a file called HERE.sh.

#!/bin/bash 
export APP_ID_HERE=your-app-id-here 
export APP_CODE_HERE=your-app-code-here

I source it into my environment with . HERE.sh to avoid any hard-coded credentials accidentally being released with my source.

Structure

The web server component will need several files you can see summarized in the listing below. Start by running mkdir -p app/api_1_0.

├── app   
|   ├── __init__.py 
│   └── api_1_0 
│       ├── __init__.py 
│       ├── demo.py 
│       ├── health.py 
├── HERE.sh 
├── manage.py 
├── config.py 
└── requirements.txt

If you aren’t using Virtual Environments for Python you should be. You can find more from the Hitchiker’s Guide to Python to get off on the right footing. You’ll want to initialize your environment with the libraries in requirements.txt which can be done with pip install -r requirements.txt if the requirements.txt contains the following dependencies.

Flask
Flask-Script
gunicorn
nltk
requests

App

We will use manage.py as the main entrypoint to our application. It looks like the following listing:

import os
import app from flask_script
import Manager, Server
app = app.create_app('default')
manager = Manager(app)

if __name__ == '__main__':
  port = os.environ('PORT', 8000)
  manager.add_command('runserver', Server(port=port))
  manager.run()

I’ve left out a few niceties like logging and printing the URL for brevity. This isn’t particularly interesting to our task and is just some housekeeping to run a simple server for our APIs.

The config.py is also important for pulling in some of those environment variables we’ll need to reference later.

import os

class Config(object):
    SECRET_KEY = os.environ.get('FLASK_SECRET_KEY')
    APP_ID_HERE = os.environ.get('APP_ID_HERE')
    APP_CODE_HERE = os.environ.get('APP_CODE_HERE')

    @staticmethod
    def init_app(app):
        pass 

config = {'default': Config}

Unlike other Python projects, our init files are pretty important on this one. In app/init.py we define the create_app function we saw in manage.py.

from config import config
from flask import Flask

def create_app(config_name):
    app = Flask(__name__)   
    app.config.from_object(config[config_name]) 
    config[config_name].init_app(app)

    from .api_1_0 import api as api_1_0_blueprint 
    app.register_blueprint(api_1_0_blueprint, url_prefix='/api/1.0') 

    return app

This gives us nice clean api versioning for any resources in our API. We also need to define app/api_1_0/init.py with some configuration

from flask import Blueprint
api = Blueprint('api', __name__)

from . import health
from . import demo

As you can see, we do need to make sure each library we create is identified as part of the blueprint.

Healthcheck

To make sure our server is running properly we can add a quick healthcheck endpoint in the file app/api_1_0/healthcheck.py.

from flask import jsonify
from flask import current_app as app
from . import api

@api.route('/health', methods=['GET'])
def handle_health():
    return jsonify({
        'hello': 'world',
        'app_id_here': app.config['APP_ID_HERE'],
        'app_code_here': app.config['APP_CODE_HERE']
        })

At this point we should be able to run python manage.py runserver and have proof of life. If you use your browser to go to http://localhost:8000/healthcheck we should get a response that confirms our server is up and has our app_id and app_code properly configured.

You may not want to display this once you hit production but is fine while we’re at a “hello world” stage.

Text

For the purposes of getting started I will use a simple text file with just the locations from before.

Venice
Mdina
Aswan
Soro
Gryfino

For more complex data sets to test with I recommend just trying out something from the New York Times, Wall Street Journal, or BBC travel sections.

Extract

We need to extract text from HTML and tokenize any words found that might be a location. We will define a method to handle requests for the resource /tokens so that we can look at each step independently.

@api.route('/tokens', methods=['GET'])
def handle_tokenize():
  # Take URL as input and fetch the body
  url = request.args.get('url')
  response = session.get('url')

  # Parse HTML from the given URL
  body = BeautifulSoup(response.content, 'html.parser')

  # Remove JavaScript and CSS from our life
  for script in body(['script', 'style']):
    script.decompose()

  text = body.get_text()

  # Ignore punctuation
  tokenizer = RegexpTokenizer(r'\w+')

  # Ignore duplicates
  tokens = set(tokenizer.tokenize(text))

  # Remove any stop words
  stop_words_set = set(stopwords.words())
  tokens = [w for w in tokens if not w in stop_words_set]

  # Now just get proper nouns
  tagged = pos_tag(tokens)
  tokens = [w for w,pos in tagged if pos in ['NNP', 'NNPS']]

  return jsonify(list(tokens))

Before this will work, we need to download NLTK resources. This is a one-time operation you can do in an interactive python shell or by executing a simple script.

$ python
...
>>> import nltk
>>> nltk.download('stopwords')
>>> nltk.download('averaged_perceptron_tagger')

With this demo.py in place we can restart our server and call the following endpoint to get back a list of any terms that could potentially be locations.

#!/bin/bash
curl http://localhost:8000/api/1.0/tokens?url=$1

For example, if you were looking at “Biking the Open Road in Colorado, With a Few Bumps Along the Way” this returns a response including terms like:

["Retreat","Industry","Boushnak","Frise","National","Mesa","Chicago","Washington","Forest","Angeles","Canyons","Colorado",...]

We’ve trimmed the wordcount down dramatically from the original article but there is still much more work that could be done to fine tune this recognition process. This is good enough for a first pass though without adding more complexity. There is obviously still some noise and not all of these are locations, but that’s what we can use the HERE Geocoder to help with.

Geocode

The HERE Geocoder API very simply takes a human understandable location and turns it into geocordinates. If you put in an address, you get back latitude and longitude.

Here’s the listing for a geocoder endpoint:

@api.route('/geocode', methods=['GET'])
def handle_geocode():
  uri = 'https://geocoder.api.here.com/6.2/geocode.json'
  headers = {}
  params = {
    'app_id': app.config['APP_ID_HERE'],
    'app_code': app.config['APP_CODE_HERE'],
    'searchtext': request.args.get('searchtext')
  }    

  response = session.get(uri, headers=headers, params=params)
  return jsonify(response.json())

Restart the python webserver and send a request for a city like “Gryfino”:

#!/bin/bash
curl http://localhost:8000/api/1.0/geocode?searchtext=$1

The response includes among other things the location I might put a marker to display this position on a map.

"DisplayPosition": { "Latitude": 53.25676, "Longitude": 14.48947 } 

Map

Finally, we’re going to take the latitude and longitude we received from our geocode request and generate a simple render with the HERE Map Image API.

This listing looks like the following:

@api.route('/mapview', methods=['GET'])
def handle_mapview():
  uri = 'https://image.maps.api.here.com/mia/1.6/mapview'
  headers = {}
  params = {
    'app_id': app.config['APP_ID_HERE'],
    'app_code': app.config['APP_CODE_HERE'],
    'poi': request.args.get('poi')
  }

  response = session.get(uri, headers=headers, params=params)
  image_path = tempfile.mktemp()
  open(image_path, 'wb').write(response.content)

  return image_path

For simplicity and brevity I haven’t included any of the error / response handling you should do here. I’ve also cheated a bit by just storing the image to the local filesystem for illustration.

Now by calling this endpoint with a comma-separated list of latitude, longitude pairs it will return a map with all of the locations having markers.

Place names without additional context can be ambiguous so in some cases there was more than one match. This map is only showing the first match, despite how much fun Venice beach may be.

Summary

The reason for making /tokens, /geocode, and /mapview separate endpoints is that this illustrates how you might setup microservices with Python + Flask for each operation you want to perform. This would allow a deployment to scale them independently.

You can find the full source code listing in the GitHub project.

For an extra dose of inception, try running the code and processing this article itself if you are still curious where to find Aswan and Gryfino.

I’m guessing that sometime in the past 5 years you’ve seen it too, so what’s going on behind the scenes and how might you use this in developing applications?

The Geolocation API

In order to make an application user-friendly, it makes good sense to know where the location of the user is. Things like places can be made more relevant, navigation and routing can be started sooner, your “things” can be found more quickly, and you don’t have to walk up-and-down all the aisles in the store looking for your mom.

If you try to solve this problem of determining the position server-side, the best indicator is generally the public IP address often reported in the X-Forwarded-For header of the HTTP request. When you are using a virtual private network (VPN) or a proxy server, that could make it look like you are 300 miles away from where you actually are. This location may be accurate enough for localization (l10n) and internationalization (i18n) use cases of country origin, but not much more than that.

You can solve it client-side too with the Geolocation API which is much more accurate. Published as a recommendation by the World Wide Web Consortium in 2013, the API allows us to write JavaScript code to fetch a position:

navigator.geolocation.getCurrentPosition(function(position) {    
     console.log(position); 
});

You can try this in a web console yourself and get back latitude and longitude values along with an approximate accuracy expressed as the radius in meters.

This makes sense for a mobile device that has a built-in GPS, but last time I checked I didn’t have a dongle for a GPS on my MacBook Pro. That made me curious about how browsers are implementing the spec.

Void Your Firefox Warranty

Firefox has a number of configuration settings you can modify to customize your experience. By typing “about:config” instead of a URL you can identify the settings for geo.wifi.uri which points to a googleapis endpoint.

I wanted to see what these requests look like so started a local webserver with http-echo-server and updated the Firefox value of geo.wifi.url to point it tohttp://localhost:8000 so I could watch the traffic.

$ npm install http-echo-server
$ http-echo-server
[server] event: listening (port: 8000)
[server] event: connection (socket#1)
[socket#1] event: resume
[socket#1] event: data
--> POST / HTTP/1.1
--> Host: localhost:8000
--> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0
--> Accept: */*
--> Accept-Language: en-US,en;q=0.5
--> Accept-Encoding: gzip, deflate
--> Content-Type: application/json; charset=UTF-8
--> Content-Length: 1198
--> Connection: keep-alive
-->
--> {"wifiAccessPoints":[{"macAddress":"81:3a:a9:88:80:58","signalStrength":-85},{"macAddress":"4c:97:6d:79:42:bf","signalStrength":-88},{"macAddress":"74:bc:0c:7f:0b:19","signalStrength":-84},{"macAddress":"00:a7:42:be:72:7f","signalStrength":-71},{"macAddress":"4c:66:6d:79:40:df","signalStrength":-85},
...]}

As you can see, Firefox is scanning the wireless local area network (WLAN) to identify the mac addresses nearby and received signal strength indication (RSSI) values and sending it all to a web service. If I make this request myself, I might get back a location like:

{
  "location": {
     "lat": 20.6842849,
     "lng": -88.567782
  },
  "accuracy": 101.0
}

This is the data that feeds back into the Geolocation API we used earlier. You can learn more about this in the Firefox docs. Chrome does the same thing but doesn’t let you choose a different provider as far as I can tell.

HERE Positioning API

If I wanted to test my web app with a specific location, I could store this static response in a file called test-location.json and set the geo.wifi.uri to point to the file on disk instead of an http endpoint.

I’d like to see a more dynamic result though and started exploring the HERE Positioning API. The HERE Positioning API works in much the same way as the default, taking in WLAN data but it also supports other radio bands — GSM, LTE, WCDMA, Cell. The Positioning API responds with JSON that Firefox expects, but I’ll need to insert a man-in-the-middle to modify the body of the request. The following listing demonstrates how I accomplished that.

I could continue to use the HERE Positioning API with Firefox, but this was more of a learning experiment than my recommendation to stand up your own service. I have other plans for how I’d like to use the Geolocation and Positioning APIs which I’ll go into detail another time.

Until then, if you have any thoughts, questions, or pointers for more information to share reach out and let me know.

This newest release candidate includes:

• verified to work on Python 3.6 and Python 2.7
• allows you to encrypt your manifest configuration files
• provides more flexibility for object initialization
• supports Predix Cache (Redis)
• supports Predix Database as a Service (PostgreSQL)
• supports Predix Event Hub
• supports UAA user management CRUD operations
• more unit and integration test coverage
• bug fixes, performance improvements, more documentation

That’s enough change that this can be called version 1.0. If you find the project to be useful, please check it out on GitHub at PredixDev/predixpy and let us know what you like or want to see from the project.

MCVE

As a best practice for asking questions on Stack Overflow you frequently should provide a Minimal, Complete, and Verifiable Example to demonstrate a problem. For the rest of this post, I’d like to provide MVCEs for solutions that PredixPy can now provide.

You can find many more examples in the PredixPy Cookbook.

Manifest Encryption

If you are pushing apps to Cloud Foundry, chances are you will have a manifest.yml file. PredixPy is flexible enough that you don’t have to store configuration details in the manifest, but it’s there so that is the default place the SDK looks for reading/writing details like uris, Predix-Zone-Id and client id/secrets.

This can make some developers uncomfortable that the manifest could be checked into a public source code repository with the credentials exposed. The Predix App Framework that is part of PredixPy now allows you to encrypt those values by adding a parameter when you initialize the Manifest.

In [1]: import predix.admin.app
In [2]: admin = predix.admin.app.Manifest(encrypted=True)

By default it will not encrypt them and you need to pass them to your app afterall anyway but you will see a warning about it.

In [3]: admin.write_manifest('manifest.secrets.yml', encrypted=False)
WARNING:root:Writing manifest manifest.yml unencrypted.

In [4]: admin.write_manifest('manifest.yml', encrypted=True)

I still wouldn’t check it into a public repository, but at least it can be encrypted while at rest.

In addition to encrypting the manifest, I get tired of coming up with client ids and secrets all the time. The docs often show the following example:

admin.create_client('client-id', 'client-secret')

These are now keyword arguments so you can continue to pass them in the original order yourself, or allow the SDK to generate a guid to use as the client id and a cryptographically-secure pseudorandom letter, digit, punctuation generator as a temporary secret that meets password guidelines.

admin.create_client()

That should clean up some of the samples that are tempting to copy and paste without changing example client ids and secrets.

Service Object Initialization

PredixPy tries to make greenfield projects easy by giving control of the whole service instance lifecycle from creation / provisioning to data ingestion and consumption. Some projects don’t need all that automation, so now there is a bit more flexibility in using any of the services from Python.

The parameters needed can be passed in as keyword arguments like the following example demonstrates:

In [1]: import predix.security.acs

In [2]: acs = predix.security.acs.AccessControl?
Init signature: predix.security.acs.AccessControl(uri=None, zone_id=None, *args, **kwargs)

You can explicitly pass whatever uri and zone_id you want to use. If you don’t pass any values, the default behavior is to:

• Check if VCAP variables are defined (as they would be in the Predix Cloud for bound services)
• Check for PredixPy environment variables, which are defined in the manifest.yml if you used PredixPy to create the service initially

This is all described in the Getting Started Concepts but works more consistently across all the supported services now.

Redis and PostgreSQL

You can now create these data management services by scripting it in Python:

import predix.admin.app
admin = predix.admin.app.Manifest()
admin.create_cache()
admin.create_dbaas()

Creating Predix Cache and Predix Database as a Service are asnchronous paid services so it can take several minutes for the instances to be provisioned and available for use. There are parameters for you to override the default names and plans if you care about those settings.

If you are unfamiliar with GoTTY, please check out my post Go Inside the Container to learn how you can setup a bash shell to work with redis-cli and psql. More importantly, you can use this bash shell to test any Python scripting you do when working with these services to help you develop faster. The PredixPy SDK primarily helps you establish the connection and everything else works the same way as the popular Python redis and sqlalchemy libraries document.

Event Hub

If you prefer to listen, you can find out a bit more about Event Hub from this introductory video:

A common use case for the service is supporting Event-Oriented Architectures that have a Pub-Sub pattern coming from machines. If you are familiar with services like RabbitMQ some of the differences with Event Hub come down to protocol and message contract.

You can get started by creating the service. You will need a UAA instance and client with appropriate scopes and authorities, but PredixPy handles those details.

import predix.admin.app
admin = predix.admin.app.Manifest()
admin.create_uaa('admin-secret-ab')
admin.create_client('client-id-bc', 'client-secret-mb')
admin.create_eventhub(publish=True, subscribe=True)

To subscribe to messages, you just listen and wait:

import predix.app
app = predix.app.Manifest()
eh = app.get_eventhub()

for msg in eh.subscriber.subscribe():
    print(msg)

To publish messages, you just queue it up and publish.

import predix.app
app = predix.app.Manifest()
eh = app.get_eventhub()
for i in range(10, 0, -1):
    msg_id = str(i)
    msg_body = 'Final Countdown {}'.format(msg_id)
    eh.publisher.add_message(msg_id, msg_body)

eh.publisher.publish_queue()

Pretty straightforward eh?

Cloud Foundry Automation

PredixPy is primarily a client library for working with Predix APIs. Since Predix Services are running in a Cloud Foundry ecosystem, the scope of the SDK creeped into things like test automation and cloud foundry management.

You can run cf commands or use the predix cli to get work done, but I am always looking for ways to automate and don’t particularly like writing bash scripts.

Here’s a collection of fun CF tricks:

import predix.admin.app
admin = predix.admin.app.Manifest()

# Get a list of marketplace services
print(admin.space.get_services())

# Create a space, name it something unique, then target it
import predix.admin.cf.spaces
predix.admin.cf.spaces.create_temp_space()
print(admin.space.name)

# Invite my team to join the fun
admin.space.org.add_user('my-team@ge.com')
print(admin.space.org.get_users())

It’s not a complete Cloud Foundry Client API but does offer some of the more common things that have come up as useful to automate.

PredixPy 1.0

That’s a taste of what’s new in PredixPy 1.0. There are still more services to be added and not every feature of the services is available yet, but it should still be able to get you through 80% of the most common use cases.

This is an open-source community SDK, so if you see something wrong — raise an issue or fork and submit a PR.

Hope that helps.

From the recent book from Ryane Bohm, Industrial IIoT for Developers, a control system is described as an arrangement of physical components that is designed to regulate the equipment to which it’s attached. This can fall into a couple of categories. The first, open-loop controls are those that operate the same way regardless of conditions. An example given is that of a washing machine running for a set amount of time, regardless of whether it cleans the clothes or not. In the case of closed-loop controls we apply sensors and the control can take action based on sensor feedback, much like a thermostat does by sensing the temperature of the room and turning on/off the furnace or air conditioning unit.

The space gets more sophisticated as you begin experimenting with Programmable Logic Controllers (PLC), Supervisory Control and Data Aquisition (SCADA), Distributed Control Systems (DCS), and Data Historians that are commonly part of the system of an industrial process. The equipment involved tends to be expensive and mission critical so when rolling out new software and applications it is not always feasible to access these systems in a test environment where you can control the parameters and validate your app.

For this reason, I experimented recently with what could be considered a poor man’s control system. I’ve hacked together a control system for my Steam Engine Digital Twin using a Belkin WeMo Insight Smart Plug. This is not what we typically mean by control system nor appropriate for an industrial environment, but for under $50 it provides an accessible means to experiment and build up a POC.

Ping

Getting things setup is not without a few obstacles which I’m not going to cover at length here. You need to setup the WeMo Mobile App to communicate with your switch and update the software.

I began using a Python API provided in a BSD licensed open-source project called ouimeaux to communicate with the switch. You can install by simply running pip install ouimeaux and pip install ouimeaux[server]. While many industrial control systems may use something like modbus or OPC-UA, the Belkin devices use basic UPnP which make the devices discoverable by a server listening on the network.

Getting started, I create an environment and call start() and discover() which will create a UPnP server listening for messages to discover the devices communicating over the network. The list_switches() call should include the WeMo I configured in the app.

import ouimeaux.environment

# Setup UPnP server and listen for devices
env = ouimeaux.environment.Environment()
env.start()
env.discover()

# Make sure we found the switch
if 'SteamControl' not in env.devices:
    print(env.list_switches())
    raise ValueError("Unable to discover control system.")

# Now we have access to our control
control = env.get_switch('SteamControl')

The Wemo switch is named SteamControl because I’m adding it to a project I’ve talked about at GlueCon and other venues. I have a Wilesco D20 Steam Engine that I refer to as my steam-powered data generator.

Source: How a steam engine works

The D20 unit is electric powered so I’m interested in learning about how much elecricity is needed to maintain pressure in the boiler and keep the machine running. I also want to respond to a situation where the pressure gets too high in the boiler to be able to shut-down power and let the system cool off or at the very least alert me to intervene and blow the whistle to stabilize the system.

Basic Insight

I have some basic control functions now over my light switch. I can find out what the current energy draw is, I can get an aggregate for the day or how long it has been enabled, and I can toggle it on or off.

# We have access to some metadata around electricity usage
print("current_power = " + str(control.current_power))
print("today_kwh = " + str(control.today_kwh))
print("today_on_time = " + str(control.today_on_time))

Mac Address, SSID, Serial Number, etc. are also useful features of the asset model for this particular device.

It is pretty easy and straightforward to send this data to Predix Time Series and Predix Asset by using the Predix Python SDK. You can find documentation on that library from the PredixPy Github.

Big Picture

The Control System is a closed-loop for providing a See-Think-Do response with milli-second level accuracy reliably in safety critical applications. The opportunity is to extract the sensor data from these Control Systems and aggregate it in the cloud where responsiveness is no longer a requirement. We can take advantage of distributed computing to apply machine learning / deep learning algorithms to help better understand and learn from the equipment for the types of insights we can use to optimize the control system and other operations in industrial environments. More about how I’m using this steam engine as an experiment in data analysis in another post.

Hope this little demonstration helps you think about the problem space, continue to learn about control systems, and maybe give the Predix Python SDK a try on your next project.