XML to JSON python script (Also JSON to XML)

Here are 2 python scripts which convert XML to JSON and JSON to XML.

XML to JSON

Create the sample XML file, with the below contents.

sample.xml
<planets>
	<planet>
		<name>
			Earth
		</name>
		<radius>
			6,371km
		</radius>
	</planet>
	<planet>
		<name>
			Jupiter
		</name>
		<radius>
			69,911km
		</radius>
	</planet>
	<planet>
		<name>
			Mars
		</name>
		<radius>
			3,390km
		</radius>
	</planet>
</planets>

Run the below python script and and it will output the converted XML as a file named output.json.

import json
import xmltodict

with open("sample.xml", 'r') as f:
	xmlString = f.read()

print("XML input (sample.xml):")
print(xmlString)
	
jsonString = json.dumps(xmltodict.parse(xmlString), indent=4)

print("\nJSON output(output.json):")
print(jsonString)

with open("output.json", 'w') as f:
	f.write(jsonString)

JSON to XML

Create the sample JSON file, with the below contents.

sample.json
{
    "planets": {
        "planet": [
            {
                "name": "Earth",
                "radius": "6,371km"
            },
            {
                "name": "Jupiter",
                "radius": "69,911km"
            },
            {
                "name": "Mars",
                "radius": "3,390km"
            }
        ]
    }
}

Run the below python script and and it will output the converted JSON as a file named output.xml.

import json
import xmltodict

with open('sample.json', 'r') as f:
	jsonString = f.read()

print('JSON input (sample.json):')
print(jsonString)

xmlString = xmltodict.unparse(json.loads(jsonString), pretty=True)

print('\nXML output(output.xml):')
print(xmlString)

with open('output.xml', 'w') as f:
	f.write(xmlString)

You may need to install the xmltodict module:

pip install xmltodict

Orbitron DDE Azimuth Elevation To Serial

Control a satellite rotator automatically from Orbitron by sending it the azimuth (or other properties) via serial.

A while ago I wrote some code to listen to Orbitron using it’s DDE inter-process comms and send the satellite information string over serial to potentially drive an aerial rotator.

I’ve since updated the code and thought I would share it here.

Using DdeOrbitronToSerial

  • Firstly install Orbitron.
    • Update 13-02-2019: Orbitron doesn’t seem to work well unless you run it as admin – it can’t save files to its install dir and won’t pick up config changes. One work-around may be a custom install location.
  • Download DDEOrbitronToSerial.zip (June-2019) and unzip it to somewhere sensible.
  • Open the install directory of Orbitron and find Setup.cfg (example path below)
    • C:\Program Files (x86)\Orbitron\Config\Setup.cfg
  • Add a similar line as below to Setup.cfg: (obviously changing paths to where you unzipped things) As per Orbitron’s documentation:

    List of available drivers. New drivers can be added by editing [Drivers] section of Config\Setup.cfg file. Example config line: MyDriver=d:\MyDriver.exe

[Drivers]
DDEOrbitronToSerial=C:\blah\DDEOrbitronToSerial.exe
  • Don’t directly run the exe, you have to launch it from Orbtron itself by going to the rotor/radio tab, selecting the correct DDE driver, in this case it’s DDEOrbitronToSerial. Then click the button to the right of the dropdown to start sending data and launch the application:

orbtron-rotor-tab

  • The DDEOrbitronToSerial application should now launch and you will be able to select your COM port and output the satellite data to the serial port.
    • NB: You can edit additional options such as com port baud rate in the config file by clicking the open config button.

Any bug reports or feature requests are welcome!

Updates

  • June 2019 – Added latest release with a bug fix:
    • FIXED: Certain USB to serial chips would only receive a few characters.

Laravel TokenMismatchException VerifyCsrfToken.php

While learning Laravel I came across the error:

laravel TokenMismatchException in VerifyCsrfToken.php line 67

This seems to be a breaking change between when the latest laracast tutorials were recorded and the version I am using.

This was due to a hidden field not being on a form which is required when using POST.

The fix was to add:


{!! csrf_field() !!}

For example:


<form method="POST" action="/cards/{{$card->id}}/notes">
<div class="form-group">
<textarea name="body" class="form-control"></textarea>
</div>

<div class="form-group">
<button type="submit" class="btn btn-primary">Add Note</button>
</div>

{!! csrf_field() !!}

</form>


 

Arduino Sleep

I needed to put an Arduino into a low power standby / sleep mode and then have it automatically wake up. I put together this Sleeper class which I have since found useful, thought I’d share it here.

A couple of notes: It uses the watchdog timer (the only timer left running when the CPU is powered down), so if you’re using a watch dog then this may not be what you’re after. Also, it’s not a very accurate timer (a few ms) so don’t use this for highly accurate timing operations – it’s fine for most applications that need to sleep / power down for a period of time and then wake up do some computing and then go back to sleep.

An example program is below, which is made up up of 3 files:

  1. The main .ino Arduino file.
  2. Sleeper.h
  3. Sleeper.cpp

Main File (example usage)


#include "Sleeper.h"

Sleeper g_sleeper;

// Initialisation code
void setup(void) {
    Serial.begin(9600);
    Serial.println("Setup done.");
}

// Main loop
void loop(void) {
    Serial.print("Total milliseconds awake: ");
    Serial.println(millis());
    delay(100);

    // Power down for 10 seconds.
    g_sleeper.SleepMillis(10000);
}


 

Download Example Project

If you’re just using this lib then you can grab the example project here. You don’t need to read the rest of this code unless you’re particularly interested, for those that are here are the .h and .cpp files:

The class I’ve put together uses the watchdog timer, so if you’re using that for you project then you’ll have to do it another way. The reason for using the watchdog timer instead of one of the normal timers in the Atmega (328p in this case for my testing) is that all normal timers are disabled when the CPU goes into the sleep mode and thus would never wake up if were waiting for one of those to tick! Hence why calling millis() will only show how long the CPU has actually been awake – it’s the time that the normal timers were ticking.

Sleeper.h


#ifndef SLEEPER_H
#define SLEEPER_H

#include <arduino.h>
#include <avr/sleep.h>
#include <avr/power.h>

// This sleeper class utilises the watchdog timer as the oscilators for the timers get powered off in SLEEP_MODE_PWR_DOWN
// mode, which is the most power effcient sleep mode. The sleep time will not be more than 16ms accurate.

class Sleeper {
    public:
    void SleepMillis(long millis);

    private:
    void DoSleep();
    void SetupWatchdog(uint8_t prescalar);

    // Watchdog Prescalars
    const int static NumberOfPrescalars = 10;
    uint8_t Prescalars[NumberOfPrescalars] = {9, 8, 7, 6, 5, 4, 3, 2, 1, 0};
    long Times[NumberOfPrescalars] = {8000, 4000, 2000, 1000, 500, 250, 128, 64, 32, 16};
};

#endif



Sleeper.cpp


#include "Sleeper.h"

   // The interupt vector for the watch dog must be present, even if empty else the CPU resets.
ISR(WDT_vect) {
    // Do nothing.
}

/// Sleeps the arduino for a number of milliseconds
void Sleeper::SleepMillis(long millis) {

    // OPTIONAL delay to wait for all things to finish, e.g. serial prints - else you may get garbled serial prints from sleeping before the sending has finished.
    //delay(50);
 
    uint8_t prescalar = 0;

    // Sleep for the longest possible watchdog timeout that's less than millis and keep going until there are no millis left.
    while (millis > Times[NumberOfPrescalars-1]) {
        for (int i = 0; i < NumberOfPrescalars; ++i) {
            if (millis > Times[i]) {
                prescalar = Prescalars[i];
                millis -= Times[i];
                break;
            }
        }
        SetupWatchdog(prescalar); 
        DoSleep();
    }
}

/// Sets up the watchdog to timeout after a certain time period.
/// There are many comments / notes in this function which have been copied directly from the data sheet for
/// user convenience.
void Sleeper::SetupWatchdog(uint8_t prescalar) {

    // Prescalars can be: 0=16ms, 1=32ms, 2=64ms, 3=125ms, 4=250ms, 5=500ms, 6=1sec, 7=2sec, 8=4sec, 9=8sec
    if (prescalar > 9) {
        prescalar = 9;
    }

    // _BV is a macro that can be thought of as a funtion that takes in a number and outputs a byte with that bit set.
 
    // WDTCSR - Watchdog Timer Control Register
    // Note that WDP[0-3] is not in order (WDP[0-2] is but WDP3 is actually bit 5 not 3!) so we have to preprocess the prescaler passed in above.
    // bits 7 = WDIF, 6 = WDIE, 5 = WDP3, 4 = WDCE, 3 = WDE, 2 = WDP2, 1 = WDP1, 0 = WDP0
 
    // WDP3 WDP2 WDP1 WDP0 Typical Time-out at VCC = 5.0V
    // 0 0 0 0 16ms
    // 0 0 0 1 32ms
    // 0 0 1 0 64ms
    // 0 0 1 1 0.125 s
    // 0 1 0 0 0.25 s
    // 0 1 0 1 0.5 s
    // 0 1 1 0 1.0 s
    // 0 1 1 1 2.0 s
    // 1 0 0 0 4.0 s
    // 1 0 0 1 8.0 s
 
    // Take the first 3 bits (WDP[0-2])
    uint8_t wdtPrescalarBits = prescalar & 7;

    // Now we need to set WDP3, to do this we don't set bit 3 but bit 5, so if our presclar had bit 8 set i.e. it 
    // was 8 or 9 being passed in then we must set WDP3 accordingly, else we could have just used prescar as it was passed in. 
    if ( prescalar & 8 ) {
        wdtPrescalarBits |= _BV(WDP3);
    }
 
    // MCUSR – MCU Status Register
    // The MCU Status Register provides information on which reset source caused an MCU reset.
    // MCUSR Bit 3 – WDRF: Watchdog System Reset Flag
    // This bit is set if a Watchdog System Reset occurs. The bit is reset by a Power-on Reset, or by writing a logic zero to the flag.
    MCUSR &= ~_BV(WDRF);
 
    // WDTCSR Bit 4 – WDCE: Watchdog Change Enable
    // This bit is used in timed sequences for changing WDE and prescaler bits. To clear the WDE bit, and/or change the prescaler bits, WDCE must be set.
    // Once written to one, hardware will clear WDCE after four clock cycles.
 
    // WDTCSR Bit 3 – WDE: Watchdog System Reset Enable
    // WDE is overridden by WDRF in MCUSR. This means that WDE is always set when WDRF is set. To clear
    // WDE, WDRF must be cleared first. This feature ensures multiple resets during conditions causing failure, and a
    // safe start-up after the failure

    // Allow changes
    WDTCSR = _BV(WDCE) | _BV(WDE);

    // WDTCSR Bit 6 – WDIE: Watchdog Interrupt Enable
    // When this bit is written to one and the I-bit in the Status Register is set, the Watchdog Interrupt is enabled. If WDE is cleared in combination with this setting, the Watchdog Timer is in Interrupt Mode, and the corresponding interrupt is executed if time-out in the Watchdog Timer occurs. If WDE is set, the Watchdog Timer is in Interrupt and System Reset Mode. The first time-out in the Watchdog Timer will set WDIF. Executing the corresponding interrupt vector will clear WDIE and WDIF automatically by hardware (the Watchdog goes to System Reset Mode). This is useful for keeping the Watchdog Timer security while using the interrupt. To stay in Interrupt and System Reset Mode, WDIE must be set after each interrupt. This should however not be done within the interrupt service routine itself, as this might compromise the safety-function of the Watchdog System Reset mode. If the interrupt is not executed before the next time-out, a System Reset will be applied.
    // Note: 1. WDTON Fuse set to "0" means programmed and "1" means unprogrammed.

    // Watchdog Timer Configuration
    //WDTON WDE WDIE Mode Action on Time-out
    //1 0 0 Stopped None
    //1 0 1 Interrupt Mode Interrupt
    //1 1 0 System Reset Mode Reset
    //1 1 1 Interrupt and System Reset Mode Interrupt, then go to System Reset Mode
    //0 x x System Reset Mode Reset
 
    // Perform the change.
    WDTCSR = _BV(WDCE) | wdtPrescalarBits | _BV(WDIE);
}

/// Powers down system.
void Sleeper::DoSleep() {

    // Set the sleep mode.
    set_sleep_mode(SLEEP_MODE_PWR_DOWN); 
    sleep_enable();

    // Put the device into sleep mode.
    sleep_mode();

    // System continues execution here after watchdog timeout.
    sleep_disable();
}

ESP8266 NodeMCU 0.9 Arduino C++

A friend from work let me borrow an ESP8266 module on a NodeMCU board and I went about trying to get started with it and thought I’d write up my notes. (using C++)

nodemcu

To make this easier I’m going to use the easily available Arduino IDE and a set of libraries for the ESP8266 module.

Install Arduino IDE

Arduino download page.

Add libraries

Using these libraries you can follow the steps below to enable your Arduino IDE to work with many ESP8266 modules. https://github.com/esp8266/Arduino

Using the Arduino IDE Boards Manager:

  • Start Arduino and open Preferences window.
  • Enter http://arduino.esp8266.com/stable/package_esp8266com_index.json into Additional Board Manager URLs field. You can add multiple URLs, separating them with commas.
  • Open Boards Manager from Tools > Board menu and install esp8266 platform.
  • Select your ESP8266 board from Tools > Board menu.

 

Code for the ESP8266 server

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WebServer.h>
#include <ESP8266mDNS.h>

const char* ssid = "YourWifiName";
const char* password = "YourWifiPassword";

void HandleRoot();
void HandleCommand();
void FlashLeds();
void TurnOffLeds();
void HandleNotFound();


ESP8266WebServer server(80);

// what to do when a client requests "http://<IP>"
void HandleRoot() {
    server.send(200, "text/plain", "hello from esp8266!"); 
}

// What to do when a client requests "http://<IP>/command"
// This function doesn't care whether it is a GET or POST at the moment, 
// it will treat them the same just to make things easier.
void HandleCommand() {
    for (uint8_t i=0; i<server.args(); i++) {
        if (server.argName(i) == "flash") {
            if (server.arg(i) == "on") {
                FlashLeds();
                server.send(200, "text/plain", "LEDs turned on");
            } else {
                TurnOffLeds();
                server.send(200, "text/plain", "LEDs turned off");
            }
        }
    }
    server.send(404, "text/plain", "Command not found");
}

void FlashLeds() {
    // Flash LEDs or something then exit this func so that the webserver carries on running.
    Serial.println("LEDs on.");
}

void TurnOffLeds() {
    // Flash LEDs or something then exit this func so that the webserver carries on running.
    Serial.println("LEDs off.");
}

// If the route is not found then default to printing out what was sent to this server to aid in debug.
void HandleNotFound() {
    String message = "File Not Found\n\n";
    message += "URI: ";
    message += server.uri();
    message += "\nMethod: ";
    message += (server.method() == HTTP_GET)?"GET":"POST";
    message += "\nArguments: ";
    message += server.args();
    message += "\n";
    for (uint8_t i=0; i<server.args(); i++) {
        message += " " + server.argName(i) + ": " + server.arg(i) + "\n";
    }
    server.send(404, "text/plain", message);}

void setup(void) {
    Serial.begin(115200);
    WiFi.begin(ssid, password);
    Serial.println("Setting up.");

    // Wait for connection
    while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
    }
    Serial.println(".");
    Serial.print("Connected to ");
    Serial.println(ssid);
    Serial.print("IP address: ");
    Serial.println(WiFi.localIP());

    if (MDNS.begin("esp8266")) {
        Serial.println("MDNS responder started");
    }

    /* 
    //I left this in commented out just to show this method as well.
    server.on("/inline", []() {
        server.send(200, "text/plain", "this works as well");
    });
    */
 
    // Setup server routes.
    server.on("/", HandleRoot);
    server.on("/command", HandleCommand);
    server.onNotFound(HandleNotFound);
    server.begin();
    Serial.println("HTTP server started");
}

void loop(void) {
    server.handleClient();
}

 

Code for webpage to send commands

The below code can be copied and pasted into a text file and then save the file as yourfilename.html, it will allow you to send commands to the module / server.

The IP address of my module is 192.168.1.115 so make sure you replace that IP in the below code with the one of your module.

Also note that you could actually serve this page straight from the module itself and not have  separate file.

<head>
    <!--<script src="jquery.js"></script>-->
    <script src="http://code.jquery.com/jquery-2.2.1.min.js "></script>

 <script>
    $(function () {

    // Bind the click of the button to the function that posts data to the server.
    $('#buttonFlashOn').click( function() {
        $.post("http://192.168.1.115/command", { flash : "on"})
         .done(function( data ) {
             //alert( "Data returned: " + data );
        });
    });

    // bind the click of the button to the function that posts data to the server.
    $('#buttonFlashOff').click( function() {
        $.post("http://192.168.1.115/command", { flash : "off"})
         .done(function( data ) {
            //alert( "Data returned: " + data );
          });
    });

});
</script>
 
 
</head>

<body>
 <br/>
 <button id="buttonFlashOn">Flash on</button> 
 <br/>
 <br/>
 <button id="buttonFlashOff">Flash off</button>
 <br/>
</body>

enjoy 🙂

Decent and cheap hosting & domains based in the UK

I’ve always had good experiences with TSO Host so I thought I’d suggest them as a good place to get started on your own web development.

They’re UK based (great for me) and all their tech support staff are as well, which is great. I’ve added a referral link above for anyone that wants to use them and possibly help me out too.

Hope that made someone’s hosting search a little easier 🙂

Laravel routes not working

If your main Laravel page is working but no other routes are then this may be of use to you.

My doc root is here:

/var/www/html/quickstart/public

Firstly enable mod rewrite:

sudo a2enmod rewrite

Then check your routes that weren’t loading, if it works great! If not, then read on.

Edit your default apache site conf file:

sudo gedit /etc/apache2/sites-enabled/000-default.conf

Mine is below:

<VirtualHost *:80>
    ServerName localhost
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html/quickstart/public
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
    <Directory "/var/www/html/quickstart/public">
        Allowoverride All
    </Directory>
</VirtualHost>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

That worked for me on Ubuntu, hope it helps someone.

I have been trying out laravel and intend to use it for some projects in the future but I’ve been finding it very frustrating to get it installed and running. The homstead vagrant setup they have just simply doesn’t work and this is the second time I’ve tried it over the past 3 months, so it wasn’t a passing bug. I’ve only ever got laravel up and running on Linux by setting it up manually and never got it working on windows. Hopefully after a few fresh installs I’ll have it down to a few nice steps and I’ll write those up =)

How to link to a section of same webpage

If you need to click an item on your page and then have it automatically scroll you to the correct location on the page then this is how you could do it:

There are two things you need:

1. A link pointing to another section of the page. (where you click)

<A HREF="#mysection">Click me to jump to blah!</A>

2. The point where the reference points to. (the part of the page you get scrolled to)

<A NAME="mysection">

… and that’s it, nice and simple!