Tuesday, 4 December 2018

Measuring function time in linux using C

Measuring Time 

So in my job recently i had to time how long it took for a module to be plugged in before that module became "ready" ... the answer was 46 seconds.. but some careful polling changes and a change how firmware was loaded mean't that got chopped in half almost to 26 seconds!

But that's not the point, the point was how did i do such timing.

The original way i found was to use to use clock(). Clock returns an approximation of the processor time used by the program.
Using the following code..

#include <time.h>
#include <stdio.h> 
int main() 
   clock_t start, stop, length; 
   start = clock(); 
   stop = clock(); 

   length = (double) stop - start / CLOCKS_PER_SEC; 

   printf("Time for your function was %ld\n", length); 

Now, this works mind you! but there's a gotchya here, oh yes, you saw that coming didn't you.. go on - admit it..
I probably didn't read the manual (RTFM!) for clock properly.... because the times i was getting back for my function calling didn't make sense compared to time stamped outputs!?
Clock() is returning time used by the program... i know - that didn't make sense, but it does if you mention the word threading.
See i was in a multi-threaded enviroment. So while this was timing my function in this "program" (ie thread!) my "program" had been running differently (in time) from other functions (ie threads!) that made up the entired module "ready" process.

Take 2. 

So instead, i used clock_gettime() with CLOCK_MONOTONIC.
This gave me a much better accuracy across my module insertion process, and started giving me numbers that made sense with other time stamped debug statements.

So.. using the following code,
#include <time.h>
#include <stdio.h>

/* Convert the timespec struct into a seconds value
static double TimeToSeconds(struct timespec* ts)
   return (double)ts->tv_sec + (double)ts->tv_nsec / 1000000000.0;

int main()
    struct timespec start, stop;
    double time_spent = 0;
    clock_gettime(CLOCK_MONOTONIC, &start);


    clock_gettime(CLOCK_MONOTONIC, &stop);
    time_spent = TimeToSeconds(&stop) - TimeToSeconds(&start);
    printf("your function time was %ld\n", time_spent);


Hopefully that helps someone with timing issues in threads etc.. lemme know if it works for you :)

Lastly - so i don't forget (possibly)... time stamping..
I'm just gonna post the code

time_t rawtime;
struct tm * timeinfo;
time ( &rawtime );
timeinfo = localtime ( &rawtime );
printf("Time Stamp is [%d:%d:%d]\n", 
        timeinfo->tm_hour, timeinfo->tm_min, timeinfo->tm_sec);

All good fun :)

Graduated!.... now what.

So you've graduated.... now what.

So that time came and went where i finished up my degree and paraded in front of my family across the stage to shake the mans hand that had never met me and get that piece of paper that now says what i always wanted to say... I'm qualified

Qualified.. I find it strange term. Its a term that says to people you know what we want you to know. But when push comes to shove..... do you?
I think the answer to that is much like the exam post, you go to an interview, the interviewer asks you the questions that we all know the answers too. So i guess... Rope interviewing? hah...

So now i've been released into the real world, to find my place, to make my 'mark' as the educators have instructed us. We have given you all the tools... we have given you the best possible start.
Sometimes it feels like i got shown to the boat, given some wood, nails, hammer and saw, and pushed off while waving goodbye i look down to find all the holes in the bottom now appearing.

Monday, 3 October 2016

Group Assignments, bad learning.

So... taking this great course (INFO263) and we are going learn up to 5 different coding languages... Just a pity that we never really get taught them.

The assignment for this course is worth like 40%, (Woah!), and it's a group assignment (Groan).

I've always thought that group assignments would be great, Really - i did... past tense..
I thought that it was a bit conveyance of what (the uni calls) the real world is all about, after all it's not like we get stuck in the corner at work and told you have 2 hours to complete this task in silence.
No - we work in teams, in groups, in lots of things like that.

However - what the real world has and our "group projects" don't have is a manager.. a boss... el captaino... big honcho .. etc etc...
This makes the real world work better, as when your fellow group member maybe not putting as much effort into the assignment as they oughta be, then in the real world.. the boss gets angry.. then turns green and smashes up the place.
Oh - wait, thats' a different boss...

Anyways - that is my point of difference is, as grumpy as you can get with different members of your group - that's all it is. Grumpiness, they don't care, all of a sudden they get a passing mark for their assignment and they've done diddly squat .. in kiwi speak - that's nothing. literally.

So anyhow - back to the topic. 5 coding languages, HTML, CSS, PHP, MYSQL, Javascript. The lecturer seems to be in his own little world with this tho, because its an INFO course, Majority of students taking it are commerce. Meaning ... it's all about the money money money (admit it - you sang that in your head didn't you).

So when he's showing us javascript on the power point, and saying (and i quote).. "I don't know why you would need all this math stuff on a business website"... its because not everything in this world is related to commerce.. thats why.
There are other reasons why you can use php other than to build a bloody shopping cart.

Anyhow - learn't more in two days of Code Academy (seriously good) than in 5 weeks of lectures, and then to top it off - the exam was all about design or proposing a solution to a website... none of which was done in class.... gah - the horror... the shame..

Anyhow - gotta run to the next class.... later gators!

Monday, 6 June 2016

Exam rants

Exams... what are they good for... really?

I realize that the academic world uses the exam process as a measure for how much information was squeezed into your brain in the last X period of time.

So they sit you down, give you a piece of paper that you quietly download your cranium onto in the hopes that in brief period of time that you are given a grade that means you know or you don't.

The problem is that.

Lets say you're like me, that sometimes a subject is interesting and that you find it invigorating to learn about it, and hence your grey matter soaks it up like a sponge to a puddle of water.
But - the person next to me finds it totally the opposite, dreary and mind numbingly aweful, and hence their brain refuses.. no matter how much they try.. to accept that this is going to be useful, and therefor prehaps you should learn it.
So - exam comes.. me.. i do really well, thats great.. person next to me does... ok - still manages to pass.

10 years pass by... (watch the tumble weeds go by)...
We both go for the same job, ohh look says mr employer... i see that you sat a course on molecular biophysics, just what we need, can you comment on ....
And here is my problem, i haven't used molecular biophysics in 10 years, i've been a truck driver... but yet - it says on my paper.. i sat an exam - and i got an X for it. At this point, i feel my knowledge is about the same as that person who was next to me....

I know - the argument here is that i was going for a job that needed that.. blah blah blah, but that aside, you see my point.

I bet for most of us, 6 months down the track - the information retained in the membrane isn't at the same level that it was during the 3 hrs that you were hurriedly scratching at a piece of paper for.

So while i can see that an exam is a measure of performance, it just sucks... esp when we get to the real world, and work in fricken TEAMS!

oh well...

Thursday, 2 June 2016

My fav little algorithm

So, just doing a bit more study and i thought i would share my favourite Algorithm so far, count sort.

I like count sort cos it's so easy and it just does it.
Count sort is non-comparison (which means it doesn't compare this number vs this number), and it's stable. This means that things are left in the order in which the appeared in the original list.

So here is how it all goes down, lets work on a list that looks like ..... alist = [2,5,3,0,2,3,0,3]

It has two stages:
Stage 1: Finding the key positions, count the appearance of that number in the list
Stage 2: Using key positions, we place the elements from the input array in the right order in the output array

To find the key positions in our list:
Find the maximum number in our list, in our example this would be.. 5
Now create a list the of [max_num+1] and fill it with zero's... so, in our case that would look like:
index   0, 1, 2, 3, 4, 5, 6
alist = [2, 5 , 3, 0, 2, 3, 0, 3]
kp=    [0, 0, 0, 0, 0, 0, 0] 

So we go thru the list and when we hit a number eg 5 we increment that index value by 1 ie...
index   0, 1, 2, 3, 4, 5, 6, 7
alist = [2, 5 , 3, 0, 2, 3, 0, 3]
kp =    [0, 0, 0, 0, 0, 1, 0] 

So when we have done that, the final array should look like
index   0, 1, 2, 3, 4, 5, 6, 7
alist = [2, 5 , 3, 0, 2, 3, 0, 3]
kp =    [2, 0, 2, 3, 0, 1, 0]

Now we have how often the numbers repeat, we need to use a running sum to organize the array to give us the true key positions we do this by... initializing our sum = 0 and our index = 0
a new c array to keep track of our sums (see what happens with that in a minute)
index    0, 1, 2, 3, 4, 5, 6
kp        [2, 0, 2, 3, 0, 1, 0]
sum = 0
index = 0
idxValue = 2

Put our sum in the array at position index position, each step here we will be incrementing index by 1
how it goes, is we get the value @ index i, switch that value for our current sum, then add the switched value to our sum value. I have tried to show this in one line, reading from left to right
index = 0  idxValue = 2 (kp[index] = 2) sum = 0 kp = [0, 0, 2, 3, 0, 1, 0]  new sum = 0+2
index = 1  idxValue = 0 (kp[index] = 0) sum = 2 kp = [022, 3, 0, 1, 0 new sum = 2+0
index = 2  idxValue = 2 (kp[index] = 2) sum = 2 kp = [022, 3, 0, 1, 0]  new sum = 2+2
index = 3  idxValue = 3 (kp[index] = 3) sum = 4 kp = [0, 2, 2, 4, 0, 1, 0]  new sum = 4+3
index = 4  idxValue = 0 (kp[index] = 0) sum = 7 kp = [0, 2, 2, 4, 0, 1, 0]  new sum = 7+0
index = 5  idxValue = 1 (kp[index] = 1) sum = 7 kp = [2, 0, 2, 4, 7, 7, 0]  new sum = 7+1

We don't go around for the last time in the array

So that gives us the key positions that the numbers should be in

Now we just lookup the value as an index and increment the value in that index
                                                                     index   0, 1, 2, 3, 4, 5, 6, 7
alist[0] = 2, kp[2] = 2 (kp = kp[2] + 1 = 3) output_list = [_, _, 2, _, _, _, _, _ ]
alist[1] = 5, kp[5] = 7 (kp = kp[5] + 1 = 8) output_list = [_, _, _, _, _, _, _, 5 ]
alist[2] = 3, kp[4] = 4 (kp = kp[4] + 1 = 5) output_list = [_, _, _, _, _, 3, _, _ ]
alist[3] = 0, kp[0] = 0 (kp = kp[0] + 1 = 1) output_list = [0, _, _, _, _, _, _, _ ]
alist[4] = 2, kp[2] = 3 (kp = kp[2] + 1 = 4) output_list = [_, _, _, 2, _, _, _, _ ]
alist[5] = 3, kp[3] = 5 (kp = kp[3] + 1 = 6) output_list = [_, _, _, _, _, _, 3, _ ]
alist[6] = 0, kp[0] = 1 (kp = kp[0] + 1 = 2) output_list = [_, 0, _, _, _, _, _, _ ]
alist[7] = 3, kp[3] = 6 (kp = kp[3] + 1 = 7) output_list = [_, _, _, _, 3, _, _, _ ]
                                                             output_list = [0, 0, 2, 2, 3, 3, 3, 5 ]

Notice in this that in red i have highlighted the two indexes that have changed, this is because we already placed a number in the position that was there, so our next number is to the right of that one.

Some code for you to try out..

def key_positions(seq, key):
    array_items = []

    for items in seq:

    # find the max in the array
    max_value = max(array_items)

    # initialise with 0's
    count = [0] * (max_value + 1)

    # count occurances of integers
    for value in array_items:
        count[value] += 1

    # make the initial sum zero
    sum = 0
    print("size of count array = " + str(max_value + 1))
    # iterate over the length of our counting array
    for idx in range(0, len(count)):
        # save the current value we have in the array
        cur_val = count[idx]
        # put the current sum to the index pos in array
        count[idx] = sum
        print("index = {0}: sum = {1} kp = {2} current_value = {3} sum-> {1}+{3}".format(idx, sum, count, cur_val))
        # add the saved value to the sum, and make that the new sum
        sum = sum + cur_val

    print("index = {0}: sum = {1} kp = {2} current_value = {3} sum-> {1}+{3}".format(idx, sum, count, cur_val))

    return count

print(key_positions([2,5,3,0,2,3,0,3], lambda x: x))

Wednesday, 1 June 2016

Quick Hull in Python

So my assignment for Algorithms was on creating an implementation for  convex hulls using the Giftwrap Algorithm, the Graham-Scan Algorithm, and also another of our choosing.

Many chose to monotone hull as their third, i thought i would give another a go, searched around a bit and came up with an implementation called Quick hull which is based around the Quicksort algorithm for those who have come across it, where a part point is formed and sorted items go on one side and the part point is incremented as it continues through the items.

I found a few posts on the Internet about how it sort of works, but thought i would post my own commentary about it and hopefully provide some help to some poor person that comes across this.

So first up Quick hull uses recursion... if you aren't a fan of recursion, or don't enjoy it... you should go here

The algorithm needs a part line to split the points in your point cloud. So we choose the minimum x value and then the maximum x value. This essentially gives us a line through which to split the points left and right on.

Once we have found that line, we now need all the points on the left hand side of the line. At this point in time we don't care about the right hand side points. So we check each point min, max, pt to see if it does a counter clockwise turn (CCW), if it does then it's on the left side of the line, so we include it in our points list.

When that is finished, we now find in that points list, the point that has the furthest distance from the line, lets call that point ptC. Every point that is now in the  triangle formed by the points min,max,ptC can be ignored since they will never be part of the convex hull.

Call our quick hull function again with the new points list and change our line end points to that of min, ptc. This will then again find the points on the left and furthest point etc, eventually leaving us with just 2 sets of points, the minimum and the maximum, of which we want the maximum because that was the last furthest point from the line. Return the max and append to the convex hull points.

Now - that does the left hand side, all we have to do is to call the quick hull function again, passing in our points list and reverse the max, min points, and append that to the previous hull list.

And we're done.

Time for some Pseudocode i think, should see what i described a bit clearer:

 Def Get Hull Points (point list):  
   Find the min x, max in list of points  
   Hull pts = Quick Hull (point list, min, max)  
   Hull pts = Hull pts + Quick Hull (point list, max, min)  
   Return Hull pts  

 Def Quick Hull (point list, min, max)  
   New pt list = Find all the points left of the line (min, max)  
   ptC = A point on the left side with the greatest distance from the line  
   if a ptC was not found  
   then return the max point  
   Hull points = Quick Hull (New pt list, min, ptC)  
   Hull points = Hull points + Quick Hull (New pt list, max, ptC)  
   Return Hull points  

Ok - now we have the Psuedocode, we need some real code.

The language i implemented this in is python (version 3), but with the Psuedoocode above you should be able to translate it into which ever code you wish.
    Quick Hull in Python
    Date: 01/06/2016
    Written by: Grant McEwan
    License: Beer license... you should buy me one if you make anything off this :)   

    This program produces a set of hull points using a similar method to the quick
    Sort Algorithm, so runs in O(n log n) time

    When called returns a list of points that forms a convex hull around 
    the listPts Given
def get_hull_points(listPts):

    # get the min, and max from the list of points
    min, max = get_min_max_x(listPts)

    hullpts = quickhull(listPts, min, max)

    hullpts = hullpts + quickhull(listPts, max, min)

    return hullpts 

    Does the sorting for the quick hull sorting algorithm
def quickhull(listPts, min, max):
    left_of_line_pts = get_points_left_of_line(min, max, listPts)

    ptC = point_max_from_line(min, max, left_of_line_pts)

    if len(ptC) < 1:
        return [max]

    hullPts = quickhull(left_of_line_pts, min, ptC)

    hullPts = hullPts + quickhull(left_of_line_pts, ptC, max)

    return hullPts

    Reterns all points that a LEFT of a line start->end
def get_points_left_of_line(start, end, listPts):
    pts = []

    for pt in listPts:
        if isCCW(start, end, pt):

    return pts

    Returns the maximum point from a line start->end
def point_max_from_line(start, end, points):
    max_dist = 0

    max_point = []

    for point in points:
        if point != start and point != end:
            dist = distance(start, end, point)
            if dist > max_dist:
                max_dist = dist
                max_point = point

    return max_point

def get_min_max_x(list_pts):
    min_x = float('inf')
    max_x = 0
    min_y = 0
    max_y = 0

    for x,y in list_pts:
        if x < min_x:
            min_x = x
            min_y = y
        if x > max_x:
            max_x = x
            max_y = y

    return [min_x,min_y], [max_x,max_y]

    Given a line of start->end, will return the distance that
    point, pt, is from the line.
def distance(start, end, pt): # pt is the point
    x1, y1 = start
    x2, y2 = end
    x0, y0 = pt
    nom = abs((y2 - y1) * x0 - (x2 - x1) * y0 + x2 * y1 - y2 * x1)
    denom = ((y2 - y1)**2 + (x2 - x1) ** 2) ** 0.5
    result = nom / denom
    return result

Wednesday, 4 May 2016

Dropbox.. do it for free

So 2nd post... I'm still wondering about what to write about on blogs, i will try not to ramble or to moan about whatever assignment that i am doing (currently writing or extending a scanner for a compiler... weeee so much fun). But no guarantees

Anyhow - Dropbox. Yes indeed

So last year dropbox was quite handy, i hadn't formally decided how to do backups of school work.. well backups full stop really, but the school work was probably the most important.
I struggled a bit too with the idea of having to write code or work on the uni computers, and then magically remember to plug a flash drive in and write to it so i could carry on at home. So i still had masses of storage left in my dropbox from when i purchased my Galaxy S3. So decided that was the best route to take, since all i had to do was login from the uni computers and then drag and drop onto the webpage and it was done. Didn't need to worry about doin' my laptop cos i had the app installed and it just came down automagically ... yes.. its what the cloud does... magic.

So - like i said i dealt with this for a year, and then recently when my extra space "time" period expired, i was suddenly faced with the dropbox being full of all the photos from my phone camera (instant upload).

Here enter google! (hah-rah!)

I pondered the thought - can i run my own dropbox, and it turns out - yes, yes i can! I found some really nifty open source software called owncloud. Looks like a great piece of work. Very fast and very nice GUI, and most importantly very easy to install.

So - hah - my server is now paying for itself :P

There's a raft of android apps (and probably apple as well) that support it. I'm still looking a bit for an app that will synchronize my camera folder to the owncloud so that when i delete something locally it disappears from up there as well. The instant upload thing is great... until you have kids that hold down the capture button... and then you have to delete 25 fotos from 2 different locations.

Early days yet, but will see how it pans out.