Mark's IT Blog – Page 24

FIT9018: Week 3

Andy Cheng presented the lecture titled “Operating Systems II: File Management” this week. In basis File Management systems allow users to store information in units called files, File systems provide a logical view for the user whilst hiding the physical view which is not interpretable by the average human.

Ordinary Files – executable, text, source code, etc.
Directories – contain Ordinary files and subdirectories (assist with Logical View)
Special Files – represent devices and other advanced file types.

We did not get into the file types in detail during the lecture or the tut however the file type and their characters (which can be view using ls -l):

– -> regular file
d -> directory file
b -> buffered special file (ie: disk drive)
c -> unbuffered special file (ie: terminal)
l -> symbolic link
p -> pipe
s -> socket

Absolute and Relative paths, self explanatory.

The lecture was cut short as we had to spent some time finishing the previous weeks slides so we did not really get into the chmod analysis which is responsible for permissions (I have had experience with this command as it is important on Unix based hosting for updating write permissions on config files). Basically the command is divided into User/Group/Others -> read, write and execute. A quick hand way for setting permissions is:

The octal (0-7) value is calculated by adding up the values for each digit

read = 4 ,write = 2, execute = 1,
User (rwx) = 4+2+1 = 7
Group(rx) = 4+1 = 5
World (rx) = 4+1 = 5
chmode mode = 0755

$ chmod 755 myfile – would be the command.

Pearl of the week: In all Unix operating systems all entities are treated as files (I assume this is related to the monolithic kernel design)

In the tut we got into working with the shell, a summary of some of the commands we covered below:

Print working directory -> pwd
Tab completes file name (or to fork where multiple possible values occur)
chmod

My favored site with listings of shell commands is: http://ss64.com/bash/ I find it the quickest and easiest for learning and look up.

To get better at command-line interface the key is regular practice. I am currently running Ubuntu on my home pc, but rarely use the shell unless testing programs. It is a good habit to get into using it.

After doing some practice it appears that using wildcard for the ls command is not very useful for find particular files in subdirectories. For example if you wanted to find all of the .ape files in your My Music folder, ls -R *.mp3 would not work. Instead one would need to use:

find ~/My Music/ -regex ".*\(mp3\)$" //find all .mp4 files
find -regex ".*\(.ape\|\.mp4\)$"  //find all .ape and .mp4 in relative directory and subdirectories

Database Technology

FIT9019: Week 3

Week 3’s lecture by Manoj Kathpalia got into database design. We continued our referencing to the ANSI/SPARC architecture which it appears will be an ongoing theme.

The lecture ran through the Database Design Cycle introducing Requirements Definition, Conceptual Design, Logical Design and Physical Design. With detailed analysis of ER modeling which applies to Conceptual and and Logical Design (which are both at the conceptual level of the ANSI/SPARC architecture).

I have some confusion over Cardinality, I assume the Cardinality of Relation and the Cardinality of a Relationship are two different things. The cardinality of a relation being the number of tuples and the cardinality of a relationship being either 1:1, 1:M or M:N. I will need to clarify this with Manoj.

The ER model is quite a simple graphical modeling tool which we review in the tutorial, it does however seem that there are a number of acceptable notations in ER modeling some of which can be seen here.

Pearl of the week:

Although ER modeling can be used for both Conceptual and Logical Design, the Conceptual Level does not include Foreign Keys and allows Many-to-Many relationships.

After working through the ER model, we touched on integrity constraints, specifically Referential Integrity constraints:

Foreign Key references must be valid or null
Update Rules either restricted, cascaded or null (ie: when the customer_ID is updated in the customer table, the foreign key in the order table can either stay the same, update or become null).
Deletion Rules either restricted, cascaded or null (ie: when the customer_ID is attempted to be deleted in the customer table, it can either be restricted (forbidden delete), delete the associated orders :O or set the customer_ID filed of the order to null).

Finally we covered an introduction to Database Design Language [DBDL]. We did some more work on DBDL in the tutorial although there were some discrepancies between the lecture example and the tut examples (ie including CK and PK). From the final question of the tutorial, my derived relations in DBDL were as follows:

book(ISBN, call_num, title, pub_date, publisher_ID)
SK title
AK call_num

author(author_ID, surname, first_name, DOB, initials)
SK surname
 SK initials

author_book(ISBN, author_ID)
FK ISBN -> book - Delete restrict, Update Restrict
FK author_ID -> author Delete restrict, Update Restrict

subject (sub_name)

subject_book (ISBN, sub_name)
FK ISBN -> book - Delete restrict, Update restrict
FK sub_name -> subject - Delete restrict, Update Restrict

publisher (publisher_ID, pub_name, street_add, city, postcode, country)
AK pub_name
SK pub_name

I did not include CK and PK as they were not included in the lecture’s example.

Systems Analysis and Design

FIT9030: Week 3

Another packed lecture from David grant for Systems Analysis and Design yesterday. The first half was a completion of last weeks unfinished lecture on Systems development methodologies. We had a more in depth look at the Object Oriented Approach and its associated methodologies, primarily iterative/spiral which is the current trend. Extreme Programming, The Unified Process and Agile Modeling (a hybrid of the two former) came next. I had not realized previously that my work place utilizes Agile Modeling with SCRUMs and SPRINTs, another timely point that related directly to my job came about when David mentioned that the Agile model would only be successful with experienced, self managing developers (haha … :|). From this lecture I became a little more confused on the distinction between models and methodologies. I recall David mentioning something on the differences but that has escaped me and when looking at wikipedia’s explanation (wiki link) it appears that they are basically substitutable terms;

‘The framework of a software development methodology consists of:
* A software development philosophy, with the approach or approaches of the software development process
* Multiple tools, models and methods, to assist in the software development process.‘

I have found the lecture and course work structure very good for this subject as it uses a consistent process; Introduction of broad concept -> Introduction of Practices -> Analysis of Practices which feels to me like conceptual clustering.

Speaking of which, following the introduction to Extreme Programming, The Unified Process and AGILE modeling, we looked at them in further detail:

Pearl of the week – Object Oriented Methodologies

Extreme Programming – Lightweight, efficient development process, utilizing informal stories and programming pairs. This methodology relies on heavy user involvement and continuous testing and integration.
Unified Process – Object Oriented, uses UML, based on (but labels 4 phases) of SDLC and Reinforces six best practices
Agile Modeling – Hybrid of XP and UP, uses Interactive and Incremental modeling. Really a logical evolution of XP and UP based on practical feedback.

The mysterious CASE tools,where introduced again in this lecture but again not covered in detail. I imagine this is because we need to understand all of the concepts in detail before using the tools but by mentioning them the topic it will not feel entirely new when we get onto it.

We completed the previous weeks lecture back on the SDLC getting some revision on the first of the 6 phases [PADIS], Planning.

Steps of the Planning phase

Defining the Problem
Confirming Feasibility [POSTEL]
Launching the Project

The current weeks lecture was a bit shorter and focused on phase 2 of the SDLC [PADIS], Analysis and Requirements Gathering. The key outcome of which was how System Requirements should be compiled; through review of documentation, interview, observation, prototypes, questionnaires, vendor research and joing application design sessions. Most of the Analysis phase is quite intuitive:

Gather information
Define System Requirements
Prioritize System Requirements
Prototype for feasibility and discovery
generate and evaluate alternatives

Also mentioned in the lecture was Business Process Re-engineering however this practice is now becoming more modular and should be driven by the business management (although a systems analyst would provide suggestions or plant seeds as the case may be).

In the tutorial we spent about 35 minutes on the Malcolm Gladwell talk, but the points where valid concluding that Systems design must pay great mind to humanistic aspects and influences.

We will start our assignments next week which should further reinforce what we have covered thus far.

http://en.wikipedia.org/wiki/Systems_Development_Life_Cycle#Requirements_gathering_and_analysishttp://en.wikipedia.org/wiki/Systems_Development_Life_Cycle#Requirements_gathering_and_analysis

Foundations of Programming

FIT9017: Week 2

Today Judy Sheard continued the introduction to the Java language whilst maintaining a conceptual anchor. Having worked through several Java text books prior to starting the course I was already familiar with most of the terms and concepts but a little revision never hurts. The topics covered in the lecture:

Classes
Fields
Assignment
Expressions
Constructors
Methods
Parameters
Accessor/Mutator methods
Data types
Display Information
Visibility Modifiers

The tutorial included some revision of the material presented in the lecture and some references to the text book which no one brought. This was a bit annoying, why they could not list the questions instead of referencing seems irrational. Similarly the decision not to make available the lecture notes prior to the lecture so students can preview it is really odd…

Anyhow, as the lecture and tut were quite basic I had some time to work on my own project in which I identified a bug. A JTable which retrieves selected data from a MySQL table and then replaces it has a bug in the update function whereby a single array variable is dropped from the end of the first row. The bug must be either in the code pulling the values from the JTable into an Array variable or the code creating an SQL update Query from the Java array:

JTable -> Array variable code:

private void updateQuoteButtonMouseReleased(java.awt.event.MouseEvent evt) {
for (int a=0;a < qData.arrayRet().length;a++)
{
for (int b=0;b < 3;b++)
{
String cString = (String) quoteTable.getValueAt(a,b);
qData.setArrayVal(a,b,cString);
}
}
try {
qData.updateBookDB();
this.qPop(Integer.parseInt(enterIDField.getText()));
} catch (SQLException ex) {
Logger.getLogger(BookPanel.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(BookPanel.class.getName()).log(Level.SEVERE, null, ex);
} catch (ClassNotFoundException ex) {
Logger.getLogger(BookPanel.class.getName()).log(Level.SEVERE, null, ex);
}
}

Array variable -> SQL UPDATE query code:

public void updateBookDB()
throws SQLException, IOException, ClassNotFoundException
{
if (qArray.length == 0)
{
SimpleDataSource.init();
Connection conn = SimpleDataSource.getConnection();
try
{
java.sql.PreparedStatement
stat2 = conn.prepareStatement
("DELETE FROM quotes WHERE book_ID = ?");
stat2.setInt(1, bookID);
stat2.execute();
}
finally
{
conn.close();
}
}
else
{
String SQLCmd = "INSERT INTO quotes values ";
for (int a=0;a < (qArray.length - 1);a++)
{
SQLCmd = SQLCmd + "(" + bookID + ", 1, " +
qArray[a][0] + ", '" +
qArray[a][1] + "', '" +
qArray[a][2] + "'),";
}
SQLCmd = SQLCmd + "(" + bookID + ", 1, " +
qArray[qArray.length - 1][0] + ", '" +
qArray[qArray.length - 1][1] + "', '" +
qArray[qArray.length - 1][2] + "');";
SimpleDataSource.init();
Connection conn = SimpleDataSource.getConnection();
try
{
java.sql.PreparedStatement
stat2 = conn.prepareStatement
("DELETE FROM quotes WHERE book_ID = ?");
stat2.setInt(1, bookID);
java.sql.PreparedStatement
stat3 = conn.prepareStatement(SQLCmd);
stat2.execute();
stat3.execute();
}
finally
{
conn.close();
}
}

After having a look at the code in notepad and being unable to find the bug. I will look for the error tomorrow night and see how long it takes me to find and fix. I won’t open another of my own projects in BlueJ after it automatically renamed my Package declarations making errors throughout my source code.

Computer Technologies and O/S

FIT9018: Week 2

Andy Cheng today presented an Introduction to Operating Systems, focusing on Unix and Unix like operating systems. This lecture was a bit short due to a time and room change. We also had to finish the previous weeks lecture with some further analysis into the steps a computer takes to process instructions. The operating system is best defined graphically (source: http://en.wikipedia.org/wiki/Operating_system <- also has a good amount of relevant details in the article):

Some key abilities of an O/S are:

Controls Hardware
Provides environment for other programs to run/compile
Controls CPU, memory and secondary memory allocation
Divides software into utilities and application depending on complexity

Modern operating systems can be defined as Multi-User and Multi-tasking (although this interrupts give the illusion of multitasking, the processor can only process one programs instruction at any given time.)

Running somewhat parallel to the abilities are, The Major Functions of an O/S:

Process Management(program/s in execution) -> creates and destroys processes, controls progress of processes, acts on exceptions/interrupts (enables multitasking), allocating hardware to processes, providing inter process communication.
File Management -> controls transfer of data to and from secondary storage devices, controlling file access, provide sharing, keep track of space and locations of files.
Memory Management -> Primary memory (RAM), closely related to process management and assists in the illusion of multitasking. Modern O/S memory management should be non-contiguous logical objects to be disjointed and to enable virtual memory (incorporating secondary memory).

The rest of the lecture was a history of Unix and the evolution of O/S, a more detailed version of which can be found at: http://en.wikipedia.org/wiki/History_of_operating_systems

The tut comprised simply of installing ubuntu onto a flash disk from which we will be completing tasks from in the future. However the tut did contain the Pearl of the week:

When installing Ubuntu onto a flash disk you must go into advanced installation options at the review stage and change the boot sector installation to the Flash disk drive (it is not altered when choosing the Flash disk as the installation directory). Without this Ubuntu will not boot from a flash disk!

Database Technology

FIT9019: Week 2

We got into the Relational Data Model today in Manoj Kathpalia’s second week lecture. I found this lecture much more interesting and challenging than last weeks which was great. Until now the terminology I had used for Relational Databases was: Table, Column, Row. Today I discovered that the RDBMS industry is engaging in mass duplication of data by having to remember the following alternate names:

Table – > Relation, Entity
Column -> Attribute / field
Row -> Record set / tuple / entry

Why everyone doesn’t just use Table, Column, Row is beyond me… We covered some properties of Relations, Primary, Alternate and Foreign Keys. I was a little confused by the Relation Languages segment where Data Definition Language [DDL] and Data Manipulation Language [DML] were outlined. Does SQL encompass both DDL and DML? If so why separate them? Further investigation reveals DDL, ‘Initially it referred to a subset of SQL, but is now used in a generic sense to refer to any formal language for describing data or information structures, like XML schemas.‘ source: http://en.wikipedia.org/wiki/Data_Definition_Language … which reminds me that I need to do some learning about XML. DDL -> Create and Delete (DBs, Tables, views, integrity contraints, indexes) DML -> Relational Calculus, Relational Algebra, Transform Oriented Languages, Graphical Languages (MS Access), 4GL, 5GL (All exhibit ‘closure ‘ property – whatever you do is not stored permanently and gone on closure of the instance.) For DML we are focusing on Relational Algebra as it is apparently more simplistic. Relational algebra has 8 basic operators:

Selection
Projection
Join
Union
Intersection
Difference
Cartesian Product
Division

We went through definitions of these with theoretical examples which I think failed to pull the theory into any relevant practical sense. I hope the tut will give me some more practical connections for this theory otherwise (without a much larger mathematical understanding) this is rote learning without the ability for application. Will update this entry after the tutorial. In the tutorial we went through some clarification of terms and then moved onto some prac questions on Relational Algebra for which I made some notepad notations:

Q4, D

[1] -> (ROOM) |x| ROOM.Hotel-No = HOTEL.Hotel-No (HOTEL)

[2] -> π type,price([1]) PROJECTION

[3] -> Θ [2].Name = Grosvenor ([2])

Q4, e

[1] -> (GUEST) |x| GUEST.Guest-No = BOOKING.Guest-No (BOOKING)

[2] -> ([1]) |x| [1].Hotel-No = HOTEL.Hotel.No (HOTEL)

[3] -> π [2].NAME, [2].ADDRESS ([2])

|x| – JOIN

π – PROJECTION

Θ – SELECTION

note that SELECTION and PROJECTION should be done first if efficiency is a concern.

Working through the Relational Algebra was good as it tied the link between theory and possible practical applications. I know if I had known this theory previously I could have saved time on several projects using SQL databases. Pearl of the Week: Relational Algebra Basic Operators:

Selection (Slices Tuples)
Projection (Dices Attributes)
Join (Natural Join, Left/Right/Full Outer Join, θ-join[uses binary operators {<, ≤, =, >, ≥}]
Union
Intersect
Difference
Cartesian Product
Division
Rename? [(Actually, Codd omitted the rename, but the compelling case for its inclusion was shown by the inventors of ISBL.)source

Systems Analysis and Design

FIT9030: Week 2

The second lecture presented by David Grant for Systems Analysis and design was focused on systems development models, primarily the Systems Development Life Cycle [SDLC]. We did not get through the whole lecture which was not surprising as it was almost 70 power point slides.

Quite interesting that we should cover this topic a week after the company I work with hit the switch on a new system upgrade for 60,000 web and email hosting clients (a chain of revealed bungles ensued with disastrous results). In the wake of confounded clients and postal resellers I can’t help but think it would have been nice if the project manager had checked out this lecture.

We started off with the SDLC as it is the foundation of development models from which most others are founded on:

P – Planning -> Define business problem and scope, produce schedule, confirm feasability, POSTEL (investigate: political, operational, schedule, technical, economic and legal as influencing factors for planning.), Resource allocation (staff and funding). Launch date.
A – Analysis -> Gather more detailed information, Define and prioritize system requirements (get client to sign these!), Build prototypes(using CASE tools), create and evaluate alternatives, Review with management.
D – Design -> network, application architecture, Interfaces, database, prototype, system controls.
I – Implement -> Construct software components (write code), test program, document system, train users, migrate legacy data, install system
S – Support -> Maintain and enhance the system, support users.

Next came some scheduling models comprising of Waterfall, Overlapping and Iteration of which I perceive Iterative to be the most applicable to all but the simplest business scenarios. The business environment is much too fluid to imagine a single launch phase.

We then investigated some approaches to system development:

Structured – Procedural/Process centric
Information Engineering -Data centric
Object Oriented – Concept based (I believe this approach is now the most popular with the rise of JAVA). As I have done most of my own programming in using the OO paradigm, the most intuitive for me.

Which brings me to the pearl of the week:

Information Engineering – A Data Centric approach to system development recognizing that the types of data created by a business is one of its most stable aspects. After hearing about this in the lecture I checked this methodology out on Wikipedia (see here).

I was interested to read that there are several variants and that the creators of this methodology have gone on to create other methodologies including an Object Oriented type. With such a huge number of methodologies and variants on each of them I understand David’s statement at the beginning of the lecture (which was something like: In practice Systems Analysts don’t actually apply these theories directly, it just helps to be aware of them). As per usual conceptual understanding/comprehension in conjunction with experience is superior to rote learning 150 models and methodologies.

Heading straight to the tutorial again after the lecture was easier this week. This week group presentations were due. We did not commit a great deal of time to preparation for this and I am glad I did not spend any more than I did. Primarily because (although interesting) the presentations were not really focused on the course scope, secondly, it was not an assessable piece of work and thirdly because we did just as well putting together our ideas over email and adjusting on the day. I agree with Linus Torvalds in his strategy of avoiding meetings and phone calls instead favoring email threads. With email threads everyone gets a turn to speak, everyone has to think before they submit their opinion and repetition is reduced. Email threads > Group meetings.

The whole tut was taken up by group presentations with intermittent additions by David, it was really enjoyable but again, not sure how related to the course it was. I look forward to doing some more practical work in the tuts as this will re-enforce the theoretical topics covered in the lectures.

Foundations of Programming

FIT9017: Week 1

A two hour lecture and two hour tut for what I expect to be my favorite subject this semester. Four hours straight from 6pm-10pm is a little tough after 4 hours of straight classes prior. The lecture was presented by Judy Sheard who was a change from the male dominated IT lecturing staff. I was a little disappointed in the opening lecture as it was very remedial, at what I would consider a high school difficulty. I must do this subject as I have had no previous ‘Official’ programming classes. I would say however that most programmers learn 90% of what they know from self learning and if that doesn’t count as credit then nothing should. Furthermore, I am the one paying for my subjects, if I want to do something more challenging then I should at least have the opportunity to do so. Judy did make a good counter to this however mentioning that the beginning of the subject may seem simple but pay attention as thing become more difficult very quickly, so lets hope she’s right.

The lecture contained administrative start (4th and final time I have to hear it :D) moved onto brief history of computing (again), brief history of programming languages and then an introduction to Java and the Object Oriented Paradigm. It was good revision but honestly, anyone who is doing postgraduate IT studies and had not already learned these basics in their preparation studies is simply not preparing sufficiently.

Luckily I found the tutorial, taken by Michael Smith to allow more independent learning at our own pace. I appreciated Michael’s frankness and self learning teaching style, he seems to take the attitude of facilitating learning for those who want to rather than dictating to the class. Look forward to more tuts in this subject.

Started using BlueJ in the tutorial which at first I thought would be far inferior to netbeans which I currently use (see link on left). I immediately found the benefits of BlueJ when creating an abstract class and a depended class, the visual layout assists with the code writing rather than interrupting it and I am sure this tool will help me fully grasp the concepts and practical implementation of Object Oriented coding.

Pearl of the week:

When writing in Java, the this.attribute notation was always a source of confusion for me… why would anyone use it? I had never needed it. The reason is specifically for when parameter names are the same as attribute names, (something I learned not to do) but a logical explanation for something that seemed quite the opposite to me previously. Thanks Michael.

Computer Technologies and O/S

FIT9018: Week 1

The first lecture for Computer Technologies and Operating systems was held today. The format for the subject is a one hour lecture followed by a three hour lab session, something I was happy to learn. Especially after finding that the lab session would be crash courses on a number of practical tasks such as setting up a web server and installing linux distros ect. The lecturer, Andy Cheng, was energetic and able to explain the topics covered in the lecture clearly.

In the lecture we, for the third time this week, went through administration topics (thankfully quickly this time). Andy went on to discuss some relevant background on computer history, followed by some fundamental computer definitions and diagrams. Things started to get interesting when we got into the definitions of:

The Bus / Bus controller – lines of for data flow between components (lines one can see on motherboards)
Fetch/Execute Cycles – Fetch retrieves next instruction from the Program Counter and loads it to in Instruction register, in Execute cycle the Control Unit decodes and executes instructions.
Interrupts – Tells the CPU to suspend current task to memory and deal with interruption task.
Multiprogramming – Multiple programs in memory and switching the processor between them using interruptions. Gives the illusion of processing many programs at once.

Unfortunately we ran short on time towards just as we were getting into the explanation of interrupts (damn the administrative crap wasting the first quarter of the lecture). This subject seems very practical and I look forward to learning many methodologies which I can apply immediately to current projects.

Database Technology

FIT9019: Week 1

Started off the semester for Database Technologies with a lecture by Manoj Kathpalia. Although the lecture was slightly unstructured we got through the admin and some basics about the history, uses and concepts of databases. Interesting points were on the evolution of databases from strings to filing cabinets to file based programs to database management systems DBMSs and the current state of relational DBMSs. I was suprised to find that we would be doing most of our coursework on Oracle systems as MySQL or another open source solution would provide the same learning environment but with greater opportunities for extended self learning.

We spent a good deal of time discussing why database processing was advantageous with the key points being:

Extracting information from data (for decision making)
Sharing of data
Standards
Controlled redundancy
Integrity control
Security
Economies of scale
Data independence

Each of these points has a number of more practical benefits for organisations.

Some disadvantages of databases primarily as cost and complexity were also raised. Although I have been using mySQL databases for some time I was not aware of the ANSI/SPARC architecture which we covered next which defined the External, Conceptual and Internal levels of modern database structure.

Pearl of the week:

The Database industry is the second largest of the IT sector, topped only by Operating systems, with over US$15 billion per year of revenues. MySQL was also not mentioned in the top DBMSs (a point that will surely change assuming Oracles purchase of Sun does not derail the c0mmunity driven project).

It will be nice to get some theoretical understanding to back up the practical experience I have had with databases, I am sure the theoretical understanding will have large influence on the way I think about and execute database design and management in the future.