Archive

Author Archive

Word Frequency Analysis program in C++

December 10, 2013 Leave a comment

I wanted to do some more work with the STL in C++, so I decided to put together a program that uses several STL containers.  I also wanted this program to do some searching and sorting of the containers so I could get a change check the performance of different STL containers.

I wrote a word and character frequency analysis program that takes two parameters, a text file that contains the text that will be parsed and analyzed and a log file that will contain the analysis.

I first looked at using a STL map, as it seemed to be close to a hash table which is the way you would write this type of algorithm in C.  But I found out that in order to sort the data by the number of occurrences of words and characters by a descending order would have forced me to perform an extra copy to a STL vector,  so I decided to use the vector from the start.

The program parses the file line by line, splitting each line into a vector of strings.   A structure is created that contains a string, for the word and a int variable to keep track of the count, each time the word is found in the text. The structure is then inserted into a vector that contains the structure  Each word is then parsed by character, and placed in a similar structure consisting of a char and a int variable that keeps track of each time the same character is discovered.

The find_if() method is used to see if the word or character already exists in the vector.  If it existed the counter is incremented, If the word or character does not exist a new structure is inserted into the vector.

In the example code I included a message file that contains the Declaration of Independence.  The program parses this file in around 1/10 of a second, Running on a 64bit Apple Mini with 8 gigs of memory.

I have placed the code out in my GitHub repository at:

https://github.com/jpitz31/Word-Frequency-Analysis

Categories: C/C++

Caesar Cipher written in C++

December 1, 2013 Leave a comment

Just submitted my last assignment for my Data Structures and Algorithms C class.  I plan on taking some time off from school so I can write some code and apply all of this great information that I have learned in the last year of attending UCSD extension program. I finished the C/C++ certification program and decided to take one more class, which was the Data Structures and Algorithms class.

Over the Thanksgiving holiday I decided to take a Caesar Cipher program I found out on the web, that I have played around with an convert it to C++.  I figure the more I write the more I will learn.

So I finished the application and posted it out on GitHub.

https://github.com/jpitz31/CaesarCipher

Instructions and an explanation of how the application functions is posted as a readme file on the GiHub site.

There is also a link to the Caesar Cipher wikipedia site.

For my C/C++ classes I used Visual Studio 2013 Express version and Professional version at work.

But at home I enjoy working on the Mac with OS X.  I have started working with Qt and Qt Creator.

I really like Qt and the cross platform environment.  So I plan on working on some C++ GUI applications  using Qt.

 

Categories: Uncategorized

Data Driven XPATH XML Parsing in C++

June 18, 2013 Leave a comment

Since I have been taking a C++ certificate course at UCSD, I have really begun to enjoy writing  C++.  At work I have been setting up InstallAnywhere scripts to build a Java application and we needed to put together a Continuous Integration build process that will run on a CRM server.

Out of the box, InstallAnywhere standard edition is pretty brain dead when it comes to automating build install executables.    The vendor wants you to spend twice the money and upgrade to the enterprise edition.

So I decided to look into writing some custom code to automate the build  process.  It turns out that InstallAnywhere creates a very large .XML file that it uses to build the install executable.

After doing some parsing and manual .XML file modification I found out that I could modify the .XML profile file that InstallAnywhere uses without impacting the execution of building the install package.

Every time we do a build we need to increment the build version, and on a the Windows version, we need to change the install location to include the version number.  This way users can install and maintain multiple versions of the same program on their workstations.

Also if the source location, where the Java application is built, ever changes we need to also modify the location where InstallAnywhere loads the application files from.

The InstallAnywhere profile .XML file is around 5400 lines long and the Windows profile has 16 different source file locations that need to changed if the build location ever changes.  There are also three different locations that relate to Installer name and install path.  These locations have to be updated with new version names.

The XML element tags are nested quite deep and using regular XML element tag looping would have been quite difficult.  So I decided use XPATH searching instead.  Since we are building the application for different OS platforms, and each OS platform has slightly different files that are required as part of the build process, different InstallAnywhere XML profile files had to be created.

I decided to use an open source project called pugixml that allows rapid, easy parsing of XML files:
Instead of creating different programs or having the program support different logic in the code to handle these XPATH differences.  I decided to create a program that would be data driven.

I created a text file that contained one line for each location to be changed,  Each line contained the full XPATH to be searched for, followed by a comma and the path to be changed within the .XML profile file.

This way I could create separate text files for each OS platform and run the program by passing in the .XML profile file and the name of the XPATH text file.

Below is a sample of the XPATH Text file, each XPATH is over 300 characters, so I will abbreviate these lines for readability.

I will also include the full C++ source code:


/InstallAnywhere_Deployment_Project/installationObjects/object/visualChildren/
object[@objectID='e40534579ff1']/.../property,C:\5.4_final\bin\components\

.../installChildren/object[@objectID='e40534589ff1']/installChildren/
object[@objectID='e40534589ff2']/installChildren/
object[@objectID='2a51ba45bc17']/property,C:\5.4_final\bin\components\

.../installChildren/object[@objectID='e40534589ff2']
/installChildren/object[@objectID='2a51ba46bc17']
/property,C:\5.4_final\bin\components\


Here is the source code:

// ****************************************
// Program: ChangePath.cpp
// Author: Joe Pitz
// Description: Updates source QTF path information of QTF
// for InstallAnywhere profile
//
// Parameters:
// Name of Profile file = QTFInstall64_0.5.3.iap_xml
// XPATH Profile File = WinXPATHProfile.txt
//
// Last Updated:
// Created Program: 06/14/2013 - Joe Pitz
//
// ***************************************

#include "pugixml.hpp"

#include <iostream>
#include <string>
#include <iostream>
#include <fstream>

using std::cout;
using std::cin;
using std::string ;
using std::ifstream ;
using std::cerr;
using std::getline;

// constants
const size_t BUFF = 255;
const int LNGBUFF = 500;
const size_t ARGCNT = 3;
const char *STRELEM = "string";
const char *SRCPATH = "sourcePath";

void modifyElement(const pugi::xml_document &doc, const char *Node, const char *Value);

void modifyElement(const pugi::xml_document &doc, const char *Node, const char *Value)
{
try
{
pugi::xpath_node_set PropNode = doc.select_nodes(Node);

for (pugi::xpath_node_set::const_iterator it = PropNode.begin(); it != PropNode.end(); ++it)
{
pugi::xpath_node node = *it;

if (strcmp(node.node().first_attribute().value(), SRCPATH) == 0)
{
node.node().child(STRELEM).text() = Value;
}
}
}
catch (std::exception ex)
{
std::cerr << ex.what();
}
}
int main(int argc , char * argv [])
{
// declare variables
size_t cnt = 0;
pugi::xml_document doc;
pugi::xml_node ObjNode;
string line;
int found;
string one;
string two;
char node[LNGBUFF];
char value[BUFF];

// check for valid command line arguments
if (argc < ARGCNT)
{
std::cerr << "* * * ERROR - Incorrect number of parameters * * * \n";
std::cerr << "Usage: " << argv [0] << " name of IntstallAnywhere File, XPATH File " << "\n";
std::cerr << "Usage: " << argv [0] << " QTFInstall64_0.5.3.iap_xml WinXPATHProfile.txt " << "\n";
return 1;
}

one = argv[1];
two = argv[2];
// open .xml file
if (!doc.load_file(argv[1]))
{
cerr << "Could not open " << argv[1] << "\n";
return -1;
}

// open file
ifstream PFile ( argv[2]);
if (PFile.is_open())
{
while ( PFile.good() )
{
// read line
getline (PFile, line);

// split line on comma
found = line.find_last_of(",");
strcpy_s(node, line.substr(0, found).c_str());
strcpy_s(value, line.substr(found + 1).c_str());

// call modifyelement
modifyElement(doc, node, value);

}
PFile.close();
}
else
{
cerr << "Unable to open file " << PFile << "\n" ;
return 1;
}

// save xml profile file
doc.save_file(argv[1]);

// debug code
// Wait for user response
cout << "\n" << "Press enter to continue...";
cin.get();
}

Categories: C/C++

Busy, but I will start posting soon

April 26, 2013 Leave a comment

Sorry, I have been tied up since last September taking classes at UCSD Extension so I can earn a Certificate in C/C++.  Too busy to work on any projects.

I did change jobs recently.  I am currently doing some consulting at Qualcomm here in San Diego working on developing some GUI automation on an application that tests ARM chip debug sub-systems.  I am making an all out effort to get up to speed on embedded systems.  With all of the connected devices, mobile devices,  and ARM processors invading our universe it is just a matter of time before all of the devices in our homes and cars will be communicating with one another.  To me this is an incredible opportunity  for software development.  I think these devices and software that talks to these devices is going to be a major portion of the software industry.  What better way to make a living than writing software that communicates with hardware.

But in between classes I did pick up a Shapeoko and have started to put it together.  I will be using GRBL loaded on an Arduino UNO and will be testing out some software and getting up to speed on learning about CNC machines and software.  As I start to make some progress I will post some articles.

I also have some robotic ideas that I am kicking around that I will post here when I get some more  time.

Until then I got to keep my nose in the C++ books.

Thanks

Joe

Categories: Uncategorized

Tutorial – Bit Banging and Boolean Math without the Math

December 9, 2012 5 comments

Overview
When I am bit banging or programming microcontrollers, sometimes I really want to focus on what the bit patterns look like, really visualize what the zeros and ones are doing. But having to do the math in many cases distracts my attention from the task at hand.

Using a calculator is slow and cumbersome, and also distracts me from the binary operations.

We humans work and think in decimal. Bit patterns are in binary. To make the conversion a bit easier we convert to hexadecimal. Having to do this conversion, either mentally, using paper and pen or calculator, at least for me, steals too much attention away from what I am really trying to visualize.

I have thought about writing a small program that will aid me in my visualization, but this too takes up time and takes me away from watching the bit patterns.

So after pondering this dilemma for a while, I came up with a quick and easy solution that give me instance binary visualization without crunching numbers, without manually converting between number bases and is instant and easy.

I normally program in C or C++ so I would like to process binary data using commands that are the same or similar to C Boolean operators.

The answer is Python. For those of you that do not know, Python is an interpreted language that has an application with an interactive like shell (command line interface) called Idle.
Using Python Idle I can quickly convert between number bases and perform all of the Boolean operations, using the same operators that I use in C or C++.

If you are new to Boolean operations or want to freshen up your skill set please refer to the following article, do not let the fact that the article is posted in the Arduino Playground put you off. The article is easy to read, clearly written and covers the topic of binary and Boolean operations quite well.

http://playground.arduino.cc/Code/BitMath

If you are new to binary and Boolean operations please feel free to use my “No Math” approach to reading this article. You too can then focus on understanding how to perform Boolean operations without having to do the math.

In my “No Math approach I will be referring to this article from time to time.

Setting up Your Environment
Python is open source and its free, Python runs on Windows, Linux and Apple OS X.
To set up your environment, head over to the following links:

http://python.org/
If you need assistance installing Python or learning about Python take a look the below link:

http://wiki.python.org/moin/BeginnersGuide

Let’s get Started
I am going to make the assumption that you know very little about python, bit banging and Boolean operations. If you are more advanced in your level of education, please feel free to skip around.

If your are new to programming, binary operations and bit banging, please feel free to Google any topic you feel you need more information on.

Read this material to gain a better understanding before continuing.

The version of Python I am using is version 3.2, if you are using this version or later versions you should be fine. If you are using a 2.x version of python, please install the current version of 3.x. You can still keep your 2.x version of Python installed.

If you would like a refresher on Binary, Decimal and Hexadecimal number systems please refer to the below link:
http://www.myhome.org/pg/numbers.html
Load the Python Idle application on your desktop. I am running Apple OS X, so my desktop images may look slightly different than yours. But the operations are the same. Once Idle loads you should see the following window:

python

Hello World

The >>> prompt is the interactive prompt where you will type your expressions.

The answer to your query will follow on the following line:

Please type the following line after the >>> prompt and press the return key:

>>> print(‘Hello World’)

You should see the following:

Hello World

>>>

Screen Shot 2012-12-09 at 11.19.20 AM

The >>> is your next prompt; you are ready for more commands and functions.

Congratulations, your have successfully written your first line of python code.

Convert Number Bases

Let’s convert some numbers between number bases:

Type in the following at the >>> prompt and press the return key:

>>> bin(100)

‘0b1100100’

>>>

Screen Shot 2012-12-09 at 11.24.11 AM

We are asking Python to convert 100 decimal to binary:

The answer we received is 1100100, notice that a 0b notation is placed at the beginning of the 1100100, this is python notation indicating that the answer is binary.  So we have converted 100 decimal to 1100100 in binary.

Type the following:

>>> hex(100)

‘0×64’

>>>

Screen Shot 2012-12-09 at 11.28.10 AM

Our answer is 64 hex, the 0x is python notation that our answer is in hexadecimal.  We have converted 100 decimal to 64 hex.

Let’s try to go the other way now, type in the following at the >>> prompt:

>>> Int(‘0×64’,16)

100

>>>

Screen Shot 2012-12-09 at 11.31.23 AM

Notice there is not a dec() function, we have to use the int() function.  Also notice

We had to place the ‘0×64’ in single quotes.  The int() function takes a string as a value.  We also have to enter a , (comma) and enter a value of 16.  The 16 tells the int() function that the 0×64 is a hexadecimal number.  The int() function converts string values to decimal values.

Now lets type the following at the >>> prompt:

>>> int(‘1100100’,2)

100

>>>

Screen Shot 2012-12-09 at 11.33.46 AM

Using the int() function we also can convert a binary string value to decimal.

The ,2 tells the int() function that we are trying to convert a binary string to a decimal number.

By using these three functions, bin(), hex, and int() we can easily convert between all number bases uses in programming.

Boolean Operations using Python

 For this section are we going to refer to the Arduino Playground article, BitMath Tutorial:

http://playground.arduino.cc/Code/BitMath

Bitwise AND Operations

Refer to the section on Bitwise AND operator.

Type in the following expressions:

>>> 0 & 0

0

>>> 0 & 1

0

>>> 1 & 0

0

>>> 1 & 1

1

>>>

Notice we have received the same answers as displayed in the Bitwise AND example.

What we have created is called a Truth table for the Bitwise AND operator.

We have displayed every possible answer that we can receive, performing the Bitwise AND operator.

Let us continue on with the Bitwise AND examples:

Type in the following at the >>> prompt:

>>> a = 92

>>> b = 101

>>> bin(a)

‘0b1011100’

>>> bin(b)

‘0b1100101’

>>> a & b

68

>>> bin(a & b)

‘0b1000100’

>>>

Here we have assigned a value of 92 to variable a and a value of 101 to variable b.  We then converted the values of a and b to binary to easily see the results of the bitwise AND operation.

We then performed the bitwise AND operation by typing a & b, this resulted in a value of 68.  But we want to visualize the answer in binary, so we typed in the bin(a & b) function.

We now can easily visualize the bitwise AND operation without having to remove our attention from looking at the binary patterns of 92 and 101 to do the math.

Let’s continue the with the bitwise AND examples:

What do you use the bitwise AND operator for in programming or bit banging?  One of the uses is to extract one of the bit values from a byte.  (eight bits)

Lets say we want to extract the value of the least significant bit of the binary number 101, the right most 1 is the least significant bit.

Type in the following at the >>> prompt:

X = 5

>>> bin(x)

‘0b101’

>>>y = x & 1

>>> bin(y)

‘0b1’

If we wanted to extract the second least significant bit from the value of 101, the second least significant bit of 101 would be the 0.

Type the following at the >>> prompt:

>>> y = x & 2

>>> bin(y)

‘0b0’

>>>

Screen Shot 2012-12-09 at 11.44.05 AM

Be careful, different processors define least significant bit positions differently.  Some processors use the left most bit as the least significant bit position and some processors use the right most bit position as the least significant bit position.

Bitwise OR Operations

 Refer to the Bitwise OR Operator section in the Arduino Playground article.

I will let you build the bitwise OR truth table using the examples in the article.

We will focus on what you can use the bitwise OR operator for.

We use the bitwise OR operator to set bits in a byte (binary number).

Let say we want to set the least significant bit in the value of 0b100.

Type in the following at the >>> prompt:

>>> x = 4

>>> bin(x)

‘0b100’

>>> y = x | 1

>>> bin(y)

‘0b101’

>>>

Now let’s set the second bit to a value of 1, ie: 111

Type in the following at the >>> prompt:

>>> y = y | 2

>>> bin(y)

‘0b111’

Screen Shot 2012-12-09 at 11.48.53 AM

Bitwise XOR Operations

 Refer to the Bitwise XOR Operator section in the Arduino Playground article.

I will let you build the bitwise XOR truth table using the example in the article.

We will focus on what you can use the bitwise XOR operator for.

We use the bitwise XOR operator to flip or toggle  bits in a number (binary number).

Let say we want to flip the least significant bit in the value of 0b110.

Type in the following at the >>> prompt:

>>> X = 6

>>> bin(x)

‘0b110’

>>> y = x ^ 1

>>> bin(y)

‘0b111’

>>>

Now let’s flip the second bit in the value 0b111

Type in the following at the >>> prompt:

>>> y = y ^ 2

>>> bin(y)

‘0b101’

>>>

Screen Shot 2012-12-09 at 11.53.09 AM

This is very cool.  We can now control setting and viewing bit values and patterns quickly and easily.

Bitwise NOT Operations

 Refer to the Bitwise NOT Operator section in the Arduino Playground article.

Feel free to play around with the NOT operator.  But study this operator carefully as zero and negative numbers are represented differently than you might think.  Take a look at what a sign bit is and read the article on two’s complement.

The main operation of the bitwise NOT operator is to flip 0 to 1 and 1 to 0.  Also this operator works on one number instead of two numbers.

Bit Shift Operators

 Refer to the Bitwise Shift Operator section in the Arduino Playground article.

Feel free to play around with both the Left Shift and Right Shift Operators.

Use the bin() function to check the binary values as we have done with the other bitwise operators.

Real use of the Left Shift and Right Operators are to pull off either the least significant or most significant bit from a number one bit at a time.

Screen Shot 2012-12-09 at 12.01.56 PM

Rest of the Article

You now have enough information to read the rest of the article on your own and apply the “No Math” Python approach.  Let’s now look at a real bit banging example.

Bit Banging Example

 Programming microcontrollers or trying to reverse engineer what a microcontroller program is doing involves processing a lot of binary information.  For example, one way in which microcontrollers communicate with other devices is to use a simple two wire interface called I2C, pronounced I squared C.

Using I2C involves sending binary data to an address of a device that is listening for requests from the microcontroller,  (master device) Once a request is received by the device, (slave) , the device sends either one or more bytes of data back to the microcontroller.

In learning how to program communications to I2C devices or trying to reverse engineer data coming from I2C devices you have to look at the data in it’s binary form.

I will provide you an easy example of how to do this using the methods you have just learned.

In learning I2C it is often wise to study other programs programmers have written so you can lean how data is passed between microcontrollers and devices.

So for our example we will look at a program snippet of some RobotC code written to bit bang an I2C device.

I have taken a sample bit banged C program written in RobotC by Xander who maintains a Robot Blog called:

http://botbench.com/blog/2011/02/20/bit-banged-i2c-master-on-robotc-for-arduino/#comment-2674

In this blog article he is bit banging an I2C program to communicate with a LED array to flash LED lights in a sequence.

We will reverse engineer the for loop he is using to send  a series of bits to the LED array.

If you have never programmed before, please Google variables, for loop, functions and while loops.

Here is the code snippet we will be looking at:

while (true) {

for (int i = 0; i < 8; i++) {

PCF8574write(~(1<<i));

wait1Msec(50);

}

}

Normally I would write a small Python program to print out the values of the for loop.  But since I am assuming everyone is new to programming and Boolean logic I will manually type out each iteration of the for loop.

Special Note:  Python does not support a byte data type, so in our bit banging example we are going to use the modulo operator to display an integer as an unsigned byte.  Please Google inf you need assistance,

For Example:  If we perform the the following example:  >>> ~(1 << 0), in Python we get -2,  If we then use >>> bin(-2), we get ‘-0b10′.  To correct the problem and show a full 8 bits type in the following:

>>> bin(-2 % 256)

’0b11111110′

In order to properly see what is going on with bits we need to see the full 8 bits.

Screen Shot 2012-12-10 at 12.15.39 PM

The While is looping forever,  ignore this for now.
The for loop loops where the variable i is equal from 0 to 7.

PCF8574write() is a custom function that we will not discuss as part of this article.  You can refer to the code to see what this function does on your own.

What we are interested in is the value of i and what binary number we pass to the PCF8574write() function.

Lets translate each iteration of the for loop:

Lets type out each iteration of the loop in Python:

The variable i starts out as 0 and ends up at a value of 7:

>>> ~(1<<0)

-2

Lets pull this expression apart.  We are performing a bitwise NOT on the following value.  We Left Shift the value of 1, i times.

That is we NOT 1 << 0.  We are pushing bits from right to left,.In the first loop we are Left Shifting 1, 0 times, or we NOT the most significant bit.

So if we look at the binary value of -2 we get:

>>> bin(~(1<<0))

‘0b10’

Lets continue, you will soon see the pattern.

>>> ~(1<<0)

-2

>>> bin(-2 % 256)

‘-0b11111110’

>>>~(1<<1)

-3

>>> bin(-3 % 256)

‘-0b11111101’

>>> ~(1<<2)

-5

>>> bin(-5 % 256)

‘-0b11111011’

>>> ~(1<<3)

-9

>>> bin(-9 % 256)

‘-0b11110111’

>>> ~(1<<4)

-17

>>> bin(-17 % 256)

‘-0b11101111’

>>> ~(1<<5)

-33

>>> bin(-33 % 256)

‘-0b11011111’

>>> ~(1<<6)

-65

>>> bin(-65 % 256)

‘0b10111111’

>>> ~(1<<7)

-129

>>> bin(-129 % 256)

‘-0b1111111’

The leading 0 is omitted in this case.

It should read ‘-0b01111111′

Screen Shot 2012-12-10 at 12.33.29 PM

Now if you have not done so already, go to the web page and watch the video:

http://botbench.com/blog/2011/02/20/bit-banged-i2c-master-on-robotc-for-arduino/#comment-2674

Now look at the bit patterns very closely.

Notice that as the for loop increases that the 0 bit keeps on moving to the left.

I will try not to confuse you too much right now, (there is another bitwise AND and a Left Shift operation in the i2c_write() function), the priority is to understand Boolean operations and how to use Python to assist you in understanding these Boolean operations.  If you feel up to it, reverse engineer the i2c_write() function to see how the bit patterns are further processed before they are sent to the LED array.

In working with microcontrollers when a bit is high, having a 1 value, normally a device is turned on.  If a bit is low, having a 0 value a device is turned off.

Notice that we are working with 8 bits and our LED array has 8 LED’s.  Yes , that is correct.  Every time we set a bit on, having a value of 1, we turn on the LED.  When we set the bit value to 0 or set the bit off, we turn off the LED.

Now watch the video again.  Now image that both for loops, one in the main function and the for loop in the i2c_write function are running as fast or faster than the LED’s are blinking.

The logic in the i2c_write() is performing a bit more complex logic,  use python to visualize how the High or Low bit states are changing as the 8 bits are being stepped through in the for loop.

Please keep on learning about microcontrollers and Boolean operations, as these are the building blocks to programming microcontrollers.

Categories: Python

Chicken Light Relay Timer

November 13, 2011 13 comments

Chicken Light Timer

The wife is into raising chickens and now that the daylight savings is getting near she wanted a way to keep the egg laying production in high gear.  According to the material that she reads, chickens will maintain their egg laying production if they get 12 – 15 hours of daylight.

I thought to myself, what a great opportunity for me to put together a DIY electronics project that would also be green.  I wanted a quick and inexpensive solution so I decided to go with the Arduino as the MCU and build everything from scratch.

This document will document my project from beginning to end.

I had picked up a battery powered LED light that would be great for this project.

I have a bunch of ATMega chips lying around as well as a few Arduino clone boards, so I was well on to my way of building this project with material and parts that are in my parts bins in my shop.

I started to think about what kind of design I wanted and decided to go simple, lean and mean.

I have done some work with latching relays in the past and really like the idea of using the MCU to latch the relay on and off without having to keep constant power on the ATMega ports all of the time.  The chicken coop is located in an area where there is no easy access to AC power and I wanted a battery solution that I could, at a later date, add solar panel access for charging.

First off I had to determine the volts and amp requirements of the LED light.

I took the LED light apart and soldered some wires to the existing switch circuit so I could trigger the light with the relay.

The LED light consumed 6 volts at around 500 milliamps.

D-cell batteries originally powered the LED light.  I did not want two different battery sources and I did not want to change batteries all of the time.  So I decided on a single battery source to power the MCU and LED light.

Below is image of LED light mounted in Chicken coop, two wires for switch and two wires for power:

Once I decided on a circuit I laid out the circuit on a breadboard using an Arduino clone as the microcontroller.  This way I could test the circuit using components that I knew worked and gave me a stable platform to write the code.

I also do not like to power external devices directly from the microcontroller ports so I decided to use some NPN transistors as part of the relay circuit.

To provide an accurate time source for the relay timer I decided to use a DS1307 RTC, Real Time Clock chip.  LadyAda has a really good write up on using this chip and has created her own code branch of JeeLab’s RTCLib library.

Here are the links to the web sites:

http://www.ladyada.net/learn/breakoutplus/ds1307rtc.html

https://github.com/adafruit/RTClib

Here is an image of the breadboard setup:

The breadboard shows a SparkFun DS1307 RTC, I have several of these that I use for testing purposes. But the production version of the Light Relay board uses the same RTC board that LadyAda documents in her writeup.

The Arduino clone pictured is the Diavolino from Evil Mad Science Labs.  Nice board and very inexpensive.  I use this board as one of my prototype boards.

Since I needed two different voltages for this configuration, 5 volts for the microcontroller and 6 volts for the LED light, I decided to use two linear power regulators.  For low cost I decided to use 7805 and 7806 regulators to drive the microcontroller and the LED light.

Since I wanted to keep the price low, I decided to build the production version of the circuit board on solder based breadboards.

After building this board, I learned a lot of lessons, In the future, I more than likely take the time to build my own custom PCB boards.  Building the solder breadboard was very time consuming and tedious.

Here is a shot of the finished production breadboard:

Notice the terminal blocks on the breadboard.  The left terminal block is connected to the two switch wires of the LED Light.  The output from the relay is connected to this terminal block.

The terminal block on the right is the voltage input for the microcontroller and relay circuit.

The 90 degree header pins are used to upload the Arduino program to the ATMega 328P.  Notice the red wire just below the right terminal block.

This wire was originally set up to connect to the USB power lead of the FTDI USB-TTL cable.

This way I could power the circuit from the USB port when I was debugging the circuit.  Just before installation I removed the lower red wire so I could power the circuit from the right terminal block from the single source battery.

My biggest expense was to purchase a deep cycle 12-volt wet cell battery.  The amp hour rating on this battery was around 110 amp hours.

This way I could power the board and light for several weeks on a single charge.  I decided to re-charge the battery once a week, until I can add the solar panel charging circuit.

After building the above circuit board and voltage regulator boards I mounted the boards into a project case.

Below is an image of the finished case with the circuit boards mounted:

Notice the top terminal block. This terminal block provides 12 volts from the deep cycle battery and powers the voltage regulators.

The lower left terminal block on the voltage regulator board provides 6 volts to power the LED light.

The right terminal block on the voltage regulator board powers the microcontroller circuit.

I also purchased a plastic battery box and mounted the micro-controller, relay circuit and the battery in this battery box just behind the chicken coop.

Here are some images of the mounting:

Here is an image of the circuit:

Here is the code for the Relay Controller:

I have included two programs that can be used as is or modified for your custom use.  One program, RTC_Relay, this program is based on turning the relay on and off within a 24 hour period when the end time is greater than the start time.

The second program, RTC_Relay_DHMS, Allows a Start Hour and a Start Minute to be set, and a day, hour, minute and second duration to be set.  When the Start Hour and Start Minute time is reached the relay is turned on and stays on for the given day, hour, minute and second duration.

The code is well documented and I have given Creative Commons license to the code, so please feel free to use the code as long as you give me credit in your code.

// Program .....: RTC_Relay
// Author ......: Joe Pitz, Objetek Systems, objetek@gmail.com
// Copyright ...: Creative Commons, CC BY-SA
// Description .: Turn on and off a latching relay using a ATMega 328P
// Date Created.: 10/30/2011
// Usage and dependencies:
//
// Variables are used to set start time and end time.
// Times are based on 24 hour clock, i.e.:
// 1:00 pm = 13:00 and Midnight is 00:00
//
// Start time must be less than end time and both times must fall within the
// same 24 hour period.
//
// Code uses RTCLIB, a branch provided from:
// https://github.com/adafruit/RTClib
//
// Thanks to LadyAda for the great tutorial on DS1307
// http://www.ladyada.net/learn/breakoutplus/ds1307rtc.html
//

#include <Wire.h>
#include "RTClib.h"

RTC_DS1307 RTC;

// Setup Start and End TImes
// This is where you define your start hour and start minute times
unsigned int StartHr = 19;
unsigned int StartMin = 58;

unsigned int StartDayofWeek;
unsigned int CurrentDay;
unsigned int EndDay;

// Sleep is a measure of minutes and must be greater than 0
// is used to pause the loop() function.  If you are checking for a start
// time of hours and minutes you do not want to use up processor
// cycles by checking the time every second. Every minute or so it good unless
// you are performing a more frequent activity.
// In this program we are turning on a relay for a period of time every day,
// so minutes are fine for a sleep delay.

long Sleep = 1L;

// The delay() function takes a argument of milliseconds so
// we need a multipler of 60,000 in order to convert our delay variable (sleep)
// to millisconnds.  So a delay of 1 minute  (1 * 60,000) = 60,000 milliseconds.  

long MultiMinute = 60000L;

boolean LightOn = false;

unsigned int EndHr = 19;
unsigned int EndMin = 59;

// ledPin used for debug using Arduino on board LED.
int ledPin = 13;

// Pins used to control latching relay
int RelayOn = 6;
int RelayOff = 7;

void setup () {
    Serial.begin(57600);
    Wire.begin();
    RTC.begin();

    pinMode(RelayOn, OUTPUT);
    pinMode(RelayOff,OUTPUT);

    RTC.adjust(DateTime(__DATE__, __TIME__));

  if (! RTC.isrunning()) {
    Serial.println("RTC is NOT running!");
    // following line sets the RTC to the date & time this sketch was compiled
    RTC.adjust(DateTime(__DATE__, __TIME__));
  }
}

void loop () {
    DateTime now = RTC.now();

    Serial.print(now.year(), DEC);
    Serial.print('/');
    Serial.print(now.month(), DEC);
    Serial.print('/');
    Serial.print(now.day(), DEC);
    Serial.print(' ');
    Serial.print(now.hour(), DEC);
    Serial.print(':');
    Serial.print(now.minute(), DEC);
    Serial.print(':');
    Serial.print(now.second(), DEC);
    Serial.println();

    // Check status of light
    if (LightOn == false)
    {
      if (((int)now.hour() >= StartHr && (int)now.minute() >= StartMin) &&
         ((int)now.hour() <= EndHr && (int)now.minute() < EndMin))
      {
        // turn on light
        LightOn = true;
        digitalWrite(RelayOn, HIGH);

        Serial.print("Light On"); 

         // only need a short time to trigger the latching relay
        //then we can turn the port back off
        delay(1000);
        digitalWrite(RelayOn, LOW); 

      }
    }
    else
    {
      // Check curren time, turn off light when conditions are met
      if ((int)now.hour() >= EndHr && (int)now.minute() >= EndMin)
      {
        LightOn = false;
        // turn light off
        digitalWrite(RelayOff, HIGH);

        Serial.print("Light Off"); 

         // only need a short time to trigger the latching relay
        //then we can turn the port back off
        delay(1000);
        digitalWrite(RelayOff, LOW);
      }
    }

    // for debugging purposes go ahead and set your delay(3000) so you
    // can debug quickly then for production you can change your delay
    // time to the calculation.
    // delay(3000);
    delay((Sleep*MultiMinute));
}
// Program .....: RTC_Relay_DHMS
// Author ......: Joe Pitz, Objetek Systems, objetek@gmail.com
// Copyright ...: Creative Commons, CC BY-SA
// Description .: Turn on and off a latching relay using a ATMega 328P
// Date Created.: 10/30/2011
// Usage and dependencies:
//
// Variables are used to set relay start time, Duration of relay on time
// is performed by setting day, hour, minute and second variables.
//
// Times are based on 24 hour clock, i.e.:
// 1:00 pm = 13:00 and Midnight is 00:00
//
// Duration can exceed 24 hour period.

//
// Code uses RTCLIB, a branch provided from:
// https://github.com/adafruit/RTClib
//
// Thanks to LadyAda for the great tutorial on DS1307
// http://www.ladyada.net/learn/breakoutplus/ds1307rtc.html
//
#include <Wire.h>
#include "RTClib.h"

RTC_DS1307 RTC;

// Setup Start and End Times
// This is where you set your relay start time

int StartHr = 16;
int StartMin = 13;

// Sleep is a measure of minutes and must be greater than 0
// is used to pause the loop() function.  If you are checking
// for a start time of hours and minutes you do not want to use
// up processor cycles by checking the time every second.
// every minute or so it good unless you are performing a more
// frequent activity.  In this program we are turning on a
// relay for a period of time every day, so minutes are fine for
// a sleep delay.

long Sleep = 1L;

// The delay() function takes a argument of milliseconds so
// we need a multipler of 60,000 in order to convert our delay
// variable (sleep) to millisconnds.  So a delay of 1 minute
// (1 * 60,000) = 60,000 milliseconds.  

long MultiMinute = 60000L;

boolean LightOn = false;

// Relay will stay latched on until current time
// matches duration variables
// If duration is 1 day and 3 hours then
// relay will stay latched on 1 day and 3 hours from the start time

DateTime future;
DateTime DelayFuture;
DateTime Start;

int DurDay = 0;
int DurHour = 0;
int DurMinute = 1;
int DurSecond = 0;

// ledPin used for debug using Arduino on board LED.
int ledPin = 13;

// Pins used to control latching relay
int RelayOn = 6;
int RelayOff = 7;

void setup () {
    Serial.begin(57600);
    Wire.begin();
    RTC.begin();

    RTC.adjust(DateTime(__DATE__, __TIME__));

  if (! RTC.isrunning()) {
    Serial.println("RTC is NOT running!");
    // following line sets the RTC to the date & time this
    // sketch was compiled
    RTC.adjust(DateTime(__DATE__, __TIME__)); 

  }

   // Start the StartHr and StartMin for Relay times
   DateTime now = RTC.now();
   DateTime SetStart(now.year(),now.month(),now.day(),StartHr,StartMin,
      now.second());

   Start = SetStart;
}

void loop () {
    DateTime now = RTC.now();

    Serial.print(now.year(), DEC);
    Serial.print('/');
    Serial.print(now.month(), DEC);
    Serial.print('/');
    Serial.print(now.day(), DEC);
    Serial.print(' ');
    Serial.print(now.hour(), DEC);
    Serial.print(':');
    Serial.print(now.minute(), DEC);
    Serial.print(':');
    Serial.print(now.second(), DEC);
    Serial.println();

     // Check status of light
    if (LightOn == false)
    {
      // Calculate current time + delay minutes, (Sleep variable time)
      // to give a wide window, making sure we can hit the start time
      // if you start the program after this window the relay will
      // not start until the StartHr and StartMin comes around
      // the next day.

      DelayFuture = CalcFuture(Start,0L,0L,Sleep,0L);

      if ((int)now.hour() >= StartHr && (int)now.hour() <=
         DelayFuture.hour() && (int)now.minute() >= StartMin &&
         (int)now.minute() <= DelayFuture.minute())
      {
        // set future DateTime, used to determine duration for
        // light on time
        future = CalcFuture(now,DurDay,DurHour,DurMinute,DurSecond);

        // turn on light
        LightOn = true;
        digitalWrite(RelayOn, HIGH);

        Serial.print("\r\nLight On\r\n"); 

        // only need a short time to trigger the latching relay
        //then we can turn the port back off
        delay(1000);
        digitalWrite(RelayOn, LOW); 

      }
    }
    else
    {
      // Check curren time, turn off light when conditions are met
      if ((int)now.day() >= (int)future.day() && (int)now.hour() >=
         (int)future.hour() && (int)now.minute() >= (int)future.minute())
      {
        LightOn = false;
        // turn light off
        digitalWrite(RelayOff, HIGH);

        Serial.print("\r\nLight Off\r\n"); 

        // only need a short time to trigger the latching relay
        //then we can turn the port back off

        delay(1000);
        digitalWrite(RelayOff, LOW);
      }
    }    

    // for debugging purposes go ahead and set your delay(3000) so
    // you can debug quickly then for production you can change
    // your delay time to the calculation.
    // delay(3000);
    delay((Sleep*MultiMinute));
} 

// CalcFuture() uses unixtime() to calculate proper DateTime for
// passed days, hours, minutes and seconds
DateTime CalcFuture(DateTime now, int Days, int Hours, int Minutes,
   int Seconds)
{
  DateTime future;
  long DaySeconds = 86400L;
  long HourSeconds = 3600L;
  long MinuteSeconds = 60L;

  future = (now.unixtime() + (Days * DaySeconds) +
     (Hours * HourSeconds) + (Minutes * MinuteSeconds) +
     Seconds);
  return future;
}
Categories: Uncategorized

Latest Cool Stuff

September 30, 2011 Leave a comment

Most of my day I spend my time writing C# and SQL code.  Since I do a lot of database data manipulation I am always running into issues with column length of data, when importing data from .csv files into database tables.  I started playing around with Python for some projects I have been working on at home.

I have found Python, extremely useful and easy to use.  I have not had a chance to use an interpretive language in many years.

So I decided to write a simple Python script to check the max length of data, in a particular column,  in a .csv file.

The script takes three arguments,  The .csv file name, the  delimiter that the .csv file is using and the number of the column (zero based)

that you want to parse.

When the script runs it parsed that column for every row in the .csv file and returns the longest length of text in the file.

The script runs very fast and parses what it is suppose to do nicely.

Python is very cool and something that I will keep on learning.

Here is the code that I wrote to parse .csv files:

#===============================================================================
# Program....: ParseCSV.py
# Author.....: Joe Pitz
# Date.......: 09/13/2011
# Description: Pass .csv file, which column to parse and delimiter
# program will return longest length of column, zero based
#===============================================================================
#
import sys
args = sys.argv[1:]

if len(args) != 3:
    print("ParseCSV takes three arguments\n\r" )
    print("<.csv file> <delimiter> <column to parse, zero based>\n\r")
    print ("Ex: ParseCSV file.csv ; 4")
    sys.exit(None)

try:
    file = args[0]
    delimiter = args[1]
    column = int(args[2])

    chrCnt = 0

    f = open(file)
    for line in f:
        llist = line.split(delimiter)

        # Check for number of columns
        numCols = len(llist)

        if column > numCols:
            print ("ERROR -> column argument is greater than number
            of columns in file <- ERROR" )
            sys.exit(None)

        colLen = len(llist[column])

        if colLen > chrCnt:
            chrCnt = colLen

    print("Longest column is " + str(chrCnt))
except TypeError:
    print "ERROR -> Check your parameters <- ERROR"
    f.close()
except StandardError:
    print "ERROR -> Error Parsing .csv File <- ERROR"
except IOError:
    print "ERROR -> Error finding or opening .csv file <- Error"
else:
    f.close()    

sys.exit(None)
Categories: Uncategorized

Consulting at Sony Electronics

November 8, 2010 Leave a comment

A few months ago I started a new consulting project.  I am working at Sony Electronics in the VAIO mobile of America group.  I was brought in to setup unit testing and white box test strategies.  But I am currently doing some ASP.NET development.

We plan on re-designing a customer survey web site using ASP.NET MVC.  So I am getting up to speed on ASP.NET MVC and starting to work on design.

Categories: Uncategorized

Publishing Articles

April 30, 2010 Leave a comment

I have been busy learning Linq to SQL and Writing articles about my data layer ventures.

Here are some links to my articles:

Article about creating data layers for ado.net
Optimize your Data Layer for quicker Code Development

Article about Linq to SQL Performance Considerations Part 1
Linq to SQL Performance Considerations Part 1

Article about Linq to SQL Performance Consideratiions Part 2
Linq to SQL Performance Considerations Part 2

Categories: Uncategorized

SQL Group by Date Range

February 20, 2010 2 comments

I recently solved an interesting problem.  SQL Server datetime column treats a calendar day as being from 12:00 am to 12:00 pm.  Where I work we run production from 03:00 am one day to 03:00 am the next day.

We recently needed to add data to our web site that reports daily production totals for the last seven days.

After doing some testing I realized that grouping by date was not going to work for two reasons.  Finished products are scanned out anytime during either of our two shifts,  Using a group by date would create individual rows every time the time portion of the date changes during a given date.  And the above  problem of when our day starts and ends.

After doing some digging I came up with the following stored procedure:

I set the starting date used in a while loop that is 9 days earlier than the current date.

For that day I construct a starting datetime variable and an ending datetime variable based on our starting shift time and ending shift time for that day.

Using a select case statement I build a date range expression that I can then use as part of my group by clause.

I insert these summed passed products into a temp table.  I them increment the date and loop again.

The while loop continues until the current date.

In the last 7 days there will always be a week-end where production sums will be null.

I can them filter the week-end data out and pull the last 7 days of data and return those values in the stored procedure.

USE [Production Monitoring]
GO
/****** Object:  StoredProcedure [dbo].[PassedScannedOut]    Script Date: 02/16/2010 16:01:07 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

– =============================================
– Author:                <Joe Pitz>
– Create date: <02/12/2010>
– Description:        Sums scanlogging by date for passed results for last 7 days

– Last Updated:

– =============================================
ALTER PROCEDURE [dbo].[PassedScannedOut]
@ScanStation varchar(50)

AS
BEGIN

Declare @StartDate as datetime
Declare @EndDate as datetime
Declare @LastWeek as datetime
Declare @Today as date
Declare @Days int
Declare @StartDateTime as datetime
Declare @EndDateTime as datetime
Declare @EndStartDateTime as datetime
Declare @StartTime varchar(12)
Declare @EndTime varchar(12)
Declare @Hours int

Create TABLE [dbo].[#ScanOutResults] (
[ScanStation] [varchar](50),
[Date] [char](10),
[Passed] [int]
)

set @Today = GETDATE()
set @Days = -9


select @StartTime = StartTime from shifts
where shiftname = ‘Shift1′

select @EndTime = StartTime,
@Hours = Hours from shifts
where shiftname = ‘Shift2′

set @LastWeek = DateAdd(dd,@Days,@Today)
set @StartDate = @LastWeek

while (@StartDate <= GETDATE())
begin

– set the begin date and end date
set @StartDateTime = Convert(datetime,@StartDate + @StartTime,20)
set @EndStartDateTime = Convert(datetime,@StartDate + @EndTime,20)
set @EndDateTime = DateAdd(hh,@Hours,@EndStartDateTime)

– Debug code to verify sums by date range
–select * from ScanLogging
–where result = ’1′ and ScanTime >= @StartDateTime and ScanTime <= @EndDateTime
–and ScanStation = @ScanStation

insert #ScanOutResults (ScanStation,Date,Passed)
select sl.Scanstation,sl.Date,sum(sl.result) Passed from
(
select case
when ScanTime >= @StartDateTime and scantime <= @EndDateTime then CONVERT(char(10),@StartDate,111)
end
as Date,ScanStation,Result
from scanlogging
where result = ’1′ and ScanTime between @StartDateTime and @EndDateTime
and ScanStation = @ScanStation) as sl
group by sl.ScanStation,sl.Date

set @StartDate = DATEADD(dd,1,@StartDate)

end

select * from
(select top 7 * from #ScanOutResults
where ScanStation is not null
order by date desc
) sl
order by date


END

Categories: SQL and stuff
Follow

Get every new post delivered to your Inbox.