2018年4月7日 星期六

ARDUINO TIME SYNC FROM NTP SERVER USING ESP8266 WIFI MODULE

ARDUINO TIME SYNC FROM NTP SERVER USING ESP8266 WIFI MODULE

Time is the unit measuring our life. Every mechanism in the world of electronics and computers is based on time. Automation as a concept is defined by intelligent actions coordinated by time or in time. When talking about automation engineering, smart projects are driven by very precise triggers and calculations based on timing and synchronization. But how can we get the perfect time without manual adjusting? One way to accomplish that is by asking an NTP Server. In this article you will find everything about how to use ESP8266 as a time source for your Arduino projects without an RTC external module.

Using WiFi ESP8266 Arduino compatible module as a Real Time Source

Note: If you don’t know what is an ESP8266 Module I recommend you to read a complete review and tutorial here and then come back.

What is NTP Server?

The NTP acronym stands for Network Time Protocol, a networking communication protocol for clock synchronization between network clients, operating since 1980’s. The NTP job is to synchronize all network participants to Coordinated Universal Time (UTC) within a few milliseconds. To accomplish this task it uses the “intersection algorithm”, an agreement algorithm invented by Keith Marlzullo to estimate accurate time from different noisy sources. NTP can maintain time with a precision under 50 milliseconds over the public Internet and under 5 milliseconds in a LAN environment. In a more friendly description I would say that a NTP is a client-server service that can be implemented by sending or receiving timestamps via UDP requests or by broadcasting / multicasting.

How can we use NTP Servers?

NTP implementations can be found in many applications. Your operating systems most probably get the time from a NTP server, time servers, databases, weather stations, brokerage and online market exchange applications also get benefits from NTP servers by requesting accurate time. In order to interrogate an NTP server your environment should be able to open an UDP connection on a local port and then send and receive UDP packages in the same network with the server. The packages received are containing multiple information like UNIX timestamp, accuracy, delay or timezone and many development environments are providing friendly methods to extract the data in a pretty objectual format.

How accurate is an NTP server?

Before knowing how accurate an NTP server can be you need to know its architecture. A NTP server is is a hierarchical, semi-layered system of levels of clocks. See below NTP strata levels:
Arduino Time Sync from NTP Server using ESP8266 WiFi module
There are many factors that can affect the accuracy of time synchronized by NTP. A visible influence can have the following:
  • Speed (latency) of the client internet connection.
  • Strata of the time server(s) chosen for synchronization.
  • Signal distance from the servers (including to and from orbiting satellites).
  • The quality and complexity of the software algorithm used.
Note: To have a better accuracy is recommended to choose servers that are physically close to your exit point (internet provider end point). This will lessen the chance that the signal is not routed up to a geostationary satellite.  Public server examples:
  • Global; www.pool.ntp.org/zone/@
  • Asia; www.pool.ntp.org/zone/asia
  • Iran; www.pool.ntp.org/zone/ir
Typically, the URL will be something like x.asia.pool.ntp.org where x = 0,1,2 or 3.


Arduino code example for ESP8266 – NTP Server pooling

In order to get data from the NTP server we need to program the ESP module to be an UDP Client. I prefer to program the ESP8266 using the Arduino IDE, but the same result can be achieved with LUA code or via AT Commands. To accomplish that with Arduino, I used two available libraries: ESP8266WiFi.h and WiFiUdp.h. First library allow and manage WiFi connections while the second handle sending and receiving UDP packages. Both libraries are available after installing the ESP8266 board from board manager in the Arduino IDE.
Replace SSID and password with your home router WiFi credentials, replace the ntpServerName with the server you want to get time from and upload the following code to your ESP. You can point a direct IP address of the NTP Server but most likely you will loose the POOLbenefits.
/********************
- www.geekstips.com
- Arduino Time Sync from NTP Server using ESP8266 WiFi module 
- Arduino code example
 ********************/

#include <ESP8266WiFi.h>
#include <WiFiUdp.h>

char ssid[] = "*************";  //  your network SSID (name)
char pass[] = "********";       // your network password


unsigned int localPort = 2390;      // local port to listen for UDP packets

/* Don't hardwire the IP address or we won't get the benefits of the pool.
 *  Lookup the IP address for the host name instead */
//IPAddress timeServer(129, 6, 15, 28); // time.nist.gov NTP server
IPAddress timeServerIP; // time.nist.gov NTP server address
const char* ntpServerName = "time.nist.gov";

const int NTP_PACKET_SIZE = 48; // NTP time stamp is in the first 48 bytes of the message

byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing packets

// A UDP instance to let us send and receive packets over UDP
WiFiUDP udp;

void setup()
{
  Serial.begin(115200);
  Serial.println();
  Serial.println();

  // We start by connecting to a WiFi network
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, pass);
  
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

  Serial.println("Starting UDP");
  udp.begin(localPort);
  Serial.print("Local port: ");
  Serial.println(udp.localPort());
}

void loop()
{
  //get a random server from the pool
  WiFi.hostByName(ntpServerName, timeServerIP); 

  sendNTPpacket(timeServerIP); // send an NTP packet to a time server
  // wait to see if a reply is available
  delay(1000);
  
  int cb = udp.parsePacket();
  if (!cb) {
    Serial.println("no packet yet");
  }
  else {
    Serial.print("packet received, length=");
    Serial.println(cb);
    // We've received a packet, read the data from it
    udp.read(packetBuffer, NTP_PACKET_SIZE); // read the packet into the buffer

    //the timestamp starts at byte 40 of the received packet and is four bytes,
    // or two words, long. First, esxtract the two words:

    unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
    unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]);
    // combine the four bytes (two words) into a long integer
    // this is NTP time (seconds since Jan 1 1900):
    unsigned long secsSince1900 = highWord << 16 | lowWord;
    Serial.print("Seconds since Jan 1 1900 = " );
    Serial.println(secsSince1900);

    // now convert NTP time into everyday time:
    Serial.print("Unix time = ");
    // Unix time starts on Jan 1 1970. In seconds, that's 2208988800:
    const unsigned long seventyYears = 2208988800UL;
    // subtract seventy years:
    unsigned long epoch = secsSince1900 - seventyYears;
    // print Unix time:
    Serial.println(epoch);


    // print the hour, minute and second:
    Serial.print("The UTC time is ");       // UTC is the time at Greenwich Meridian (GMT)
    Serial.print((epoch  % 86400L) / 3600); // print the hour (86400 equals secs per day)
    Serial.print(':');
    if ( ((epoch % 3600) / 60) < 10 ) {
      // In the first 10 minutes of each hour, we'll want a leading '0'
      Serial.print('0');
    }
    Serial.print((epoch  % 3600) / 60); // print the minute (3600 equals secs per minute)
    Serial.print(':');
    if ( (epoch % 60) < 10 ) {
      // In the first 10 seconds of each minute, we'll want a leading '0'
      Serial.print('0');
    }
    Serial.println(epoch % 60); // print the second
  }
  // wait ten seconds before asking for the time again
  delay(10000);
}

// send an NTP request to the time server at the given address
unsigned long sendNTPpacket(IPAddress& address)
{
  Serial.println("sending NTP packet...");
  // set all bytes in the buffer to 0
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  // Initialize values needed to form NTP request
  // (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12]  = 49;
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;

  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp:
  udp.beginPacket(address, 123); //NTP requests are to port 123
  udp.write(packetBuffer, NTP_PACKET_SIZE);
  udp.endPacket();
}
This code example was inspired from here. Open your Serial monitoring terminal and reset the ESP module. You should see a result like this:
Arduino Time Sync from NTP Server using ESP8266 WiFi module
In this example I used a global NTP server (time.nist.gov), but being from Europe it is not the best choice. For my projects I use an NTP Server which is closer to my physically location like in the following example:
const char* ntpServerName = "0.europe.pool.ntp.org";
To get the NTP that match your location go to www.pool.ntp.org and in the right sidebar click on your zone / continent to see a complete list:
Arduino Time Sync from NTP Server using ESP8266 WiFi module
Arduino Time Sync from NTP Server using ESP8266 WiFi module

Configuring ESP8266 as a Time source

There are several ways to use the ESP module as a time source, but we have to choose one with the best performance during the transaction, in order to reduce or eliminate the additional time. The fastest way, and the way I used in my projects, is to get the time from the ESP8266 via serial terminals. Another way is to setup a web server on the ESP and return the time via HTTP response on demand, but this is network dependent and requires a network connected project. Keep in mind that any delay created by the output transaction will directly affect the accuracy of the time returned.
In the following example we will connect the ESP8266 with a standard Arduino UNO R3 via Serial terminals, and setup the Arduino UNO internal RTC to synchronize with the NTP given time. To do that, we will have to use the SoftwareSerial library in order to emulate serial communication on the digital GPIOs 2 and 3 and reserve this line for listening the ESP module.
Arduino Time Sync from NTP Server using ESP8266 WiFi module
As you probably know from the previous tutorials, I strongly recommend you to use a solid external 3.3v power supply for ESP8266, but for the sake of demonstration we will use UNO’s 3.3v VCC. Before doing anything on your Arduino UNO, first we should cleanup the code we just uploaded on ESP8266.
Because the Arduino Time library gives us the possibility to synchronize the internal or external RTC only by passing the UNIX time parameter, we wont need the other Serial output used in example, so the production code for ESP module should look like this:
/********************
- www.geekstips.com
- Arduino Time Sync from NTP Server using ESP8266 WiFi module 
- Arduino code example
 ********************/

#include <ESP8266WiFi.h>
#include <WiFiUdp.h>

char ssid[] = "***********";  //  your network SSID (name)
char pass[] = "***********";       // your network password


unsigned int localPort = 2390;      // local port to listen for UDP packets
IPAddress timeServerIP;
const char* ntpServerName = "time.nist.gov";

const int NTP_PACKET_SIZE = 48;

byte packetBuffer[ NTP_PACKET_SIZE];
WiFiUDP udp;

void setup()
{
  Serial.begin(115200);
  WiFi.begin(ssid, pass);
  
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
  }
  udp.begin(localPort);
}

void loop()
{
  WiFi.hostByName(ntpServerName, timeServerIP); 

  sendNTPpacket(timeServerIP);
  delay(1000);
  
  int cb = udp.parsePacket();
  if (!cb) {
    delay(1);
  }
  else {
    udp.read(packetBuffer, NTP_PACKET_SIZE); // read the packet into the buffer
    unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
    unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]);
    unsigned long secsSince1900 = highWord << 16 | lowWord;
    const unsigned long seventyYears = 2208988800UL;
    unsigned long epoch = secsSince1900 - seventyYears;
    Serial.print("UNX");
    Serial.println(epoch);
  }
  delay(10000);
}

unsigned long sendNTPpacket(IPAddress& address)
{
  Serial.println("sending NTP packet...");
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  packetBuffer[12]  = 49;
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;
  udp.beginPacket(address, 123);
  udp.write(packetBuffer, NTP_PACKET_SIZE);
  udp.endPacket();
}
As you can see we only print the UNIX time (epoch), because we will use it to pass as a parameter to the Arduino internal clock setup configuration. I also printed a string “UNX” token before the timestamp just to be sure that ESP doesn’t gives us garbage data.
Now, on the Arduino UNO, we need to listen for the ESP8266 serial transmission. When the ESP sends data on the TX serial channel, we need to check if what is coming is the actual UNIX time and not some garbage or errors. We can do that by checking the first 3 characters which should be  = “UNX” token. If this is true, then the next 10 characters should represent the UNIX epoch time. See the demonstration in the example below:
/********************
- www.geekstips.com
- Arduino Time Sync from NTP Server using ESP8266 WiFi module
 ********************/
#include <SoftwareSerial.h>
#include <TimeLib.h>

// setup a commong baudrate for UNO and ESP
int baudRate = 115200;
char unixString[11];
long unixTime;
boolean dataSync = false;
// setup the ESP emulated serial channel
SoftwareSerial esp8266(2, 3);

void setup() {
  Serial.begin(baudRate);
  esp8266.begin(baudRate);
}

void loop() {
  
  char buffer[40];
  int i = 0;

  // while the ESP output available, push it
  // into the buffer and set the flag true
  while (esp8266.available()) {
    buffer[i++] = esp8266.read();
    dataSync = true;
  }

  // if data is available, parse it
  if (dataSync == true) {
    if ((buffer[0] == 'U') && (buffer[1] == 'N') && (buffer[2] == 'X')) {
      // if data sent is the UNX token, take it
      unixString[0] = buffer[3];
      unixString[1] = buffer[4];
      unixString[2] = buffer[5];
      unixString[3] = buffer[6];
      unixString[4] = buffer[7];
      unixString[5] = buffer[8];
      unixString[6] = buffer[9];
      unixString[7] = buffer[10];
      unixString[8] = buffer[11];
      unixString[9] = buffer[12];
      unixString[10] = '\0';

      // print the UNX time on the UNO serial
      Serial.println();
      Serial.print("TIME FROM ESP: ");
      Serial.print(unixString[0]);
      Serial.print(unixString[1]);
      Serial.print(unixString[2]);
      Serial.print(unixString[3]);
      Serial.print(unixString[4]);
      Serial.print(unixString[5]);
      Serial.print(unixString[6]);
      Serial.print(unixString[7]);
      Serial.print(unixString[8]);
      Serial.print(unixString[9]);
      Serial.println();
      
      unixTime = atol(unixString);
      // Synchronize the time with the internal clock
      // for external use RTC.setTime();
      setTime(unixTime);
      dataSync = false;
    }
  }
}
The setTime() method has many overloading versions and can be used with UNIX timestamp parameter but also can take as parameters specific time parts like hour, minutes, seconds. Documentation extras from Arduino official:
/********************
- www.geekstips.com
- Arduino Time Sync from NTP Server using ESP8266 WiFi module 
- Arduino code example
 ********************/ 

 setTime(t);              // Set the system time to the
                          // give time t UNX timestamp
 setTime(hr,min,sec,day,month,yr); // Another way to set
                                   // the time with time parts
 adjustTime(adjustment); // Adjust system time by adding
                         // the adjustment value
                         // WARNING: this is not compatible with using a sync provider as it
                         // only adjusts the library system time and not time in the sync provider.
                         // This offset adjustment will be lost on the next time sync.
                         // If the time is moved forward, an immediate call to now() can get the time
                         // from the sync provider rather than the adjusted library system time.
                         // i.e. do not depend on now() to return an adjusted time when using
                         // a sync provider even when called immediately after adjustTime()

 timeStatus();   // Indicates if time has been set and
                 //  recently synchronized
                 //  returns one of the following
                 //  enumerations:
 * timeNotSet    // The time has never been set,
                 //  the clock started at Jan 1 1970
 * timeNeedsSync // The time had been set but a sync
                 //  attempt did not succeed
 * timeSet       // The time is set and is synced
                 //  Time and Date values are not valid if
                 //  the status is timeNotSet. Otherwise
                 //  values can be used but the returned
                 //  time may have drifted if the status is
                 //  timeNeedsSync.    

 setSyncProvider(getTimeFunction);// Set the external time
                                  //  provider
 setSyncInterval(interval);    // Set the number of
                               //  seconds between re-sync
After updating the time with UNIX timestamp, you can check the results by calling a function like in the following example:
void displayCurrentTime() {
  Serial.print(hour());
  Serial.print(":");
  Serial.print(minute());
  Serial.print(":");
  Serial.print(second());
  Serial.println();
  delay(1000);
}
I used the exact same principle to make a self adjusting digital clock in my house and is working pretty well so far. I also implemented few fail-proof for those situations when requests are failing or the internet connection is dead. You should make some checks and test if the difference between UNIX time received from the NTP and the current timestamp from the RTC isn’t much bigger than the delay between the requests sent to the Server. For example you can save in a variable the last UNIX timestampreceived from the ESP module and compare with the current one. The difference between them should be ~ equal with the interval used in ESP loop structure for delay.
This is how my digital clock project looks like:
Arduino Time Sync from NTP Server using ESP8266 WiFi module
It displays the clock from an NTP server on a 16×2 common LCD display, using big custom fonts, and also displays the indoor temperature and humidity on a SSD1306 OLED display measured by a DHT22 sensor. I update the time every 10 minutes and make a correction off 2 seconds each time to have a perfect GMT match.
If you are interested to get parts for your project here is what I’ve used (links from Amazon.com):
This is just a way to take advantage of an NTP in your Internet of things projects. You can probably find allot of other methods more complex or maybe better coded to accomplish this task. Also, there are quite many UDP libraries and examples out there, nothing stops you to try them and find the best solution that fits to your project. At the end of the day you should end up getting a good accurate time reference over the internet without doing manually adjustments.
Hoping that this article inspired you, i kindly invite you share this article,  subscribe my YouTube channel and join the communities on social networks. Feel free to comment or send suggestions / remarks so i can improve the content quality.


沒有留言:

張貼留言

2024年4月24日 星期三 Node-Red Dashboard UI Template + AngularJS 參考 AngularJS教學 --2

 2024年4月24日 星期三 Node-Red Dashboard UI Template + AngularJS 參考 AngularJS教學 --2 AngularJS 實例 <!DOCTYPE html> <html> <head> &...