Untangling the Knot

[Pyrotex]

Originally Posted by Symbology

I was given some additional insight today...I'm not sure what the deeper lesson is there yet. But I'm sure it's coming....

Yes, little symbolhopper,
the deeper lesson is this:
learn to control the inputs,
and you become master of the outputs.

Pyro

[Overdog]

Originally Posted by Symbology

And the data lesson learned there is that the said table itself identifies a series of paired relationships, more than just relying on the other tables. IE it may have hidden higher importance.

I'm not sure what the deeper lesson is there yet. But I'm sure it's coming.

It sounds like you are describing an "Intersection" or "Union" table.

Those relationship lines you were moving around define a 1 to many relationship between two tables. Child table has a foriegn key column which contains the value of a unique row in the Parent table (usually the "Primary Key").

But suppose you have two tables that have a Many to Many relationship. How would you set that up?

You create a Union table that goes between the two Parent tables.

For example, say you have Companies and People tables. Now say you have an Employee table. An employee is a Person, right? So their data goes in the People table. "Employee" is not an entity in and of itself as one might think, but is actually a relationship between People and Companies. So you set up the relationships 1 to many from People to Employee, and 1 to many from Companies to Employee.

Now you have a Many - Many relationship between Companies and People. A single person can be mapped to more than one company by adding multiple rows in the Employee table, and a Company can be mapped to more than one Person in the same way.

Other data that might be hidden in the Employee table would be something like "Hire Date" for example. Data relevant only to the relationship....

[Symbology]

Originally Posted by Pyrotex

Yes, little symbolhopper,
the deeper lesson is this:
learn to control the inputs,
and you become master of the outputs.

Pyro

Ah! The Corollary to

Who controls the past, controls the future
Who controls the present, controls the past.
-1984

And that also ties in with my martial arts study of the word "Before". As in move before their fist gets there.

Thanx Pyro!

[Symbology]

Originally Posted by Overdog

It sounds like you are describing an "Intersection" or "Union" table.

Those relationship lines you were moving around define a 1 to many relationship between two tables. Child table has a foriegn key column which contains the value of a unique row in the Parent table (usually the "Primary Key").

But suppose you have two tables that have a Many to Many relationship. How would you set that up?

You create a Union table that goes between the two Parent tables.

For example, say you have Companies and People tables. Now say you have an Employee table. An employee is a Person, right? So their data goes in the People table. "Employee" is not an entity in and of itself as one might think, but is actually a relationship between People and Companies. So you set up the relationships 1 to many from People to Employee, and 1 to many from Companies to Employee.

Now you have a Many - Many relationship between Companies and People. A single person can be mapped to more than one company by adding multiple rows in the Employee table, and a Company can be mapped to more than one Person in the same way.

Other data that might be hidden in the Employee table would be something like "Hire Date" for example. Data relevant only to the relationship....

Yes - Thanks Overdog. Would you agree that such things tend to be "meta data" - data about the data. And that it acts metaphorically like scaffolding to hold the once tangled lines apart so that things can move in and out of them?

[Symbology]

So in the last two weeks I have tried something new. It takes several hours of documentation, but it then allows for virtual planning in my code instead of direct manipulation.

In this particular set of problems (aka Tangled Knot) I have been assigned, there are about 60 different calls to the same function with varying input patterns.

First I create a spreadsheet listing out all the calling functions and then painstakingly recorded the inputs.

Second I color coded the inputs to their sources so that variables coming from the same source class were of the same color.



Then I could treat the whole problem as a Rubik's Cube (metaphorically) and rearrange the rows and columns in Excel until I found consistent patterns in the calls.

Then (for this particular problem) I was able to create wrapper functions that consolidated all the redundant calls. I was also able to rearrange the order of the original function arguments to put the least used optional arguments in the back and also greatly reduce the footprints of the original calls.

So using that approach I took out about 400 lines of code.



I got three lessons out of that:

1)

Weeks of coding can save hours of planning

2) That whole Rubik's Cube trick might be pretty useful for other problems - basically tag the categories, and then work with the abstract categories to reduce the apparent complexity of the problem.

Complexity is just a disarrangement of simplicity.
- George Abell
(former UCLA Aerospace Engineering professor)

3) The PFM is in the the ability to hide stuff that is still there (optional arguments) and still can be used behind the scenes and pulled out at the drop of a hat when desired.

3a) There is of course, the alternative:

[Pyrotex]

Originally Posted by Symbology

Yes - Thanks Overdog. Would you agree that such things tend to be "meta data" - data about the data. And that it acts metaphorically like scaffolding to hold the once tangled lines apart so that things can move in and out of them?

AHA!! You have opened up an amphora of vermicelli, Little Meta-Hopper.

You cannot understand the DATA until you understand the META-DATA. Mdata describes all aspects of the data's organization, key-orientations, value discretions, uniquosities, table relationships, data types, constraints, limits, groups, mortastats, fotticytes and befuddling pins.

Mdata doesn't just serve as "scaffolding" to the data, it gives MEANING to the data. To say that the data "wvy035a" is a unique key infuses it with meaning that cannot be gleaned otherwise. Now you have some power over an entire record of data that you did not previously have.

And THIS is why we design, build and speak of "databases" -- so that we may wield POWER over the apparently random and meaningless strings of naughts and unities that we use to capture the Past. For this enables us to recall and (indeed) manipulate the Past -- thereby giving us MASTERY OVER THE FUTURE!!!

BWAHAHAHAHAHAHAHA!!!!!!!!!!!!!!!!!!!!!!!!!!

[Symbology]

-- Deleted... rewritten in the next entry

[Symbology]

So I have been in this new position for about 4 months now. I have been diligently documenting their enormous network of applications, processes, and their expert system's main code. Their change process alone takes up 8+ hours of overhead per problem record using 5 major change control systems.

Most all of it is a complete patchwork. It is a combination of managers not allocating enough time to clean up and redo, as well as some programmers complete lack of interest in the concept of code reuse. So every time I step into a new function, I see yet again another variation of the same ol same ol - (Same S#!% Different Database).

One of the scariest aspects is the almost complete lack of use/understanding of Instance level methods. Everything is written as static (class level) methods that have 40-100 lines of code using the same pointer variable. I have begun recoding some of those, and hope that they will serve as a template to follow. But the challenge of training such important Object Oriented concepts to non-CS Majors is a challenge to introduce and still protect egos.

To be humble to Superiors is Duty,
to Equals Courtesy,
to Inferiors Nobleness
~ Benjamin Franklin

^ Personally I find the irony of identifying someone as "Inferior" and still being humble in their presence to be a humorous conundrum.


In one case I got permission to recode 40 function calls. I took out about 400 lines of redundant code in the process. But the culture shock, testing cost, and impact of me not knowing some particular impacts on the code have made it a VERY expensive change (about $10k+ in programmer time).

Ultimately I think it will be viewed as worthwhile. But right now it teeters on the "is it worth it" scale for me as well as everyone else.

This is about the 5th time I have taken on a major cleanup like this in as many years. And each time I have been more successful, primarily because my ability to document (and therefore explain) has improved each time.

On the other side of the lesson fence though - my martial arts and Eastern philosophy study constantly advocate "Letting it go" - "Being willing to live in Chaos", "Roll with the punches" - and of course the all important Star Trek philosophy "Resistance is Futile!"

I still remain bound and determined to find the optimal balance point that can still manage to transform a F#@ing Mess of a knot into something that has some appearance of order and consistency.

So any positive and/or critical thoughts beyond what has already been offered remain appreciated.

[Buffy]

Congratulations on biting off something you could chew without choking!

Now the trick is to move that little project from the "was it worth it" column into the "thank God he did that" column, which will take a LOT of marketing within the team, most importantly, convincing someone--quite frankly anyone on the team--to lead the meme....

Oh, what a tangled web we weave when first we practice to believe,
Buffy

[Pyrotex]

Originally Posted by Symbology

So I have been in this new position for about 4 months now.
...So any positive and/or critical thoughts beyond what has already been offered remain appreciated.

Yes.
Take care of yourself.
Never work more than 60 hours a week.
Listen to your body.
When it yawns, take a nap.
When it thirsts, drink water.
When it hungers, feed it food worthy of a code warrior.
When it hurts or cannot function at 100%, listen to it.
Keep the mind rested and inspired, or it will not be there for you when you need it most.
Let go all attachments, especially attachments to your own invincibility.
Life is short, so prioritize, prioritize, prioritize.
Enjoy what you are doing--if it becomes burdensome, it is not worth doing.
In the end, when you face eternity, it will not ask you how much code you reused, or what your cost/benefit ratio was, or how smart you were -- it will ask you only two things: have you loved? have you lived in despair?