Categories
Grid Computing Uni notes

FIT5164: Summary

FIT5164 Grid Computing was the most complex subject I have taken thus far. It was also the most interesting due to the broad range of applications that were discussed for this technology. The subject was particularly good because of the instant grid on VMware which enabled us to practically apply the complex theoretical concepts discussed in the lectures.

Dr Asad Khan presented the lectures with the assumption that students were reading up on concepts and understanding all of the tutorial work. This was necessary due to the complexity and number of topics covered. I found the challenging lecture structure as good motivation to learn as much as possible thus was very happy with it.

The first 7 weeks of the course comprised of the examinable material. This material included introductions to grid and their applications then drilling into the key concepts and components of grids. The final 3 weeks introduced some very interesting topics which were not examinable, including Wireless Sensor Networks and distributed pattern recognition. In particular, distributed pattern recognition appears to be a very interesting application for grid computing which I felt topped the course of perfectly. I would recommend this subject to anyone in the MIT/MAIT streams, however the prerequisites should include data communications and a UNIX based subject.

Categories
Grid Computing

FIT5164: Week 9

Grid computing hit its stride in week 9 😀 haha, had to happen sooner or later.
Distributed Pattern Recognition was put forth as a prime application for computing grids. Purposes could include data mining, medical research and an array of scientific and engineering pursuits. Asad Khan proposed the use of the Distributed Pattern Recognition Architecture in conjunction with distributed pattern recognition algorithms to enable recognition computation on scales far greater than current solutions permit.
In the modern world where data held by the human race is growing exponentially, the ability to draw valuable information from the mass of data is quite exciting.

Distributed hierarchical graph neuron architecture
Categories
Grid Computing

FIT5164: Week 8

Week 8 of grid computing encompassed a completion the previous lecture on Grid Resource management (specifically distributed resource management [DRM]). The week 8 lecture was then a detailed introduction to sensor grid networks which I imagine is one of the most likely futures for grid computing.

Diagramatic representation of a sensor network grid (source: week 8 lecture notes)

In sensor network Data filtering and security were the main issues discussed.

Categories
Grid Computing

FIT5164: Week 7

I am still struggling to follow the lecture material in Grid Computing, but none-the-less learning a lot and finding my feet in the more practical tutorials.

Week 7’s lecture discussed:

  • Grid Resources
  • Resource Specification
  • Grid Processing
  • Grid Application Hosting

ALthough I don't have a solid understanding of web service, I can see this architecture makes sense

Categories
Grid Computing

FIT5164: Week 6

Grid computing’s 6th week provided a look into clusters, specifically:

  • Clusters vs Grids
  • Benchmarking techniques
  • Cluster and Grid programming environments
clusters can represent 1 node in the grid

Discussion of a cluster implementation method, Beowulf described how some clusters differ. A key characteristic of clusters is their interconnect technologies, some of the options are:

  • Fast Ethernet
  • Gigabit Ethernet
  • 10 Gigabit Ethernet
  • Myrianet
  • Infiniband

For a detailed list and comparison: http://en.wikipedia.org/wiki/List_of_device_bandwidths#Local_area_networks

It is also worth noting that latency can become more important than bandwidth is many cluster networks. This would be dependent on the programs running on the cluster.

Categories
Grid Computing

FIT5164: Week 5

Week 5 of grid computing comprised of a lecture focusing on data transfer protocols for the grid environment. The tutorial returned to the Grid Security Infrastructure [GSI] covered in the previous weeks lecture.

The lecture covered 4 protocols for data transfer within a grid:

  • GridFTP
  • SCTP
  • Fast TCP
  • Sensor networks

GridFTP allows for 3rd party transfers which is the major practical difference apart from transfer speed.

gridFTP enables 3rd parties to move data around the grid

We delved into the methods by which these protocols provide advantages over standard file transfer. One question this raised in my mind was why these protocols aren’t the standard if they are superior. I guess the answer to this question lies in the need for most server to try and limit the amount of bandwidth the provide to individuals.

The first assignment for this subject is due this week, I found the specification a little bit vague and am concerned that my report will not hold enough technical details. The lack of technical detail has revealed that my depth of knowledge for this subject is not very sound so it will be a priority over the mid semester break to get some books and clarify my understanding.

Categories
Grid Computing

FIT5164: Week 4

Grid Computing’s week 4 lecture took a security theme covering the Grid Security Infrastructure [GSI], Public key infrastructure , Digital certificates, Mutual authentication, My Proxy and shibboleth.

GSI (see: http://en.wikipedia.org/wiki/Grid_Security_Infrastructure) is an overlay on the transport security protocol (SSL) utilizing asymmetric encryption and the public key infrastructure to acheive:

  • Authentication
  • Data integrity verification
  • Single sign-on
  • Inter-organisation decentralized security

All grid entities (user and processes) must have a public key certificate, for more info on public key certificates see: http://en.wikipedia.org/wiki/Public_key_certificate

GSI uses the X.509 standard which included 4 primary pieces of information:

  • subject name
  • public key
  • identity
  • digital signature

An illustration of the public key infrastructure process:

source: week 4 lecture notes

Scenario 1 -> privacy, only user can decrypt incoming data

Scenario 2 -> authentication, receivers decrypt data using the sources public key this ensures the data is coming from the correct source

Certificate authorities are required to ensure validity of public and private keys that make the users digital certificate

Categories
Grid Computing

FIT5164: Week 3

Grid Computing, week 3, saw another jam-packed lecture that left me with a great deal of questions. I think this is a good thing as it forces students to do required readings or simply be left behind in the lectures. Unfortunately I have not been keeping up with readings on this subject and am feeling the pressure now.

The lecture focused on a possible real life example of a butterfly grid. When developing such a grid, the key considerations must be identified:

  • Latency
  • Downtime
  • Security (cheating)
  • Physical scalability
  • Specific stakeholder needs (Production company, vendors, ISP, gamers, etc)

Soruce: http://pmlab.iecs.fcu.edu.tw/~cyt/cyt/data/GridApplication.ppt

The butterfly grid add 7 major tiers in the architecture… There was no diagram in the lecture to assist in understanding where these layers fit, a simplified representation can be seen above.

The layers listed in the lecture slide were:

  • Object Management System [OMS]
  • Network Protocol Stack (UPD/IP instead of TCP/IP for reduced latency)
  • Gateway Servers
  • Daemon Controllers
  • Game Servers
  • Data Store and Grid Service – existing layers
  • Globus Toolkit Services

Some news article relating to butterfly grids from early 2000s:

http://www.gridcomputingplanet.com/news/article.php/1107421/IBM-Butterflynet-Announce-Video-Game-Grid.htm

https://www-304.ibm.com/easyaccess/ukcomms/gclcontent/gcl_xmlid/91353

There is however very little recent  information on Butterfly grids to be found on the web.

Categories
Grid Computing

FIT5164: Week 2

Week two led my first lecture and tutorial for grid computing. In the lecture there were a great deal acronyms which I did not know. So, to start with here are some of the acronyms explained:

Globus Toolkit Acronyms:

VMware Workstation is utilized in this subject over VirtualBox

The tutorial enabled us to create a virtual grid setup. Where were provided with several Knoppix VMware images prepared with Globus Toolkit installed. Booting them up and getting a feel for the environment seemed to be the main point of the tut.

Categories
Grid Computing

FIT5164: Week 1

Again affected by my timetable changes I missed the first lecture on this subject. Apperently the MUSO system is still being used here so it took some tracking down but did manage to find the subjects resources. Unit resources: http://users.monash.edu.au/~asadk/units/fit5164/, see MUSO for username and password.

A quick summary from the lecture notes of lecture 1:

Unit leader: Dr Asad Khan

Grids enable ‘On-Demand’ computing, examples of grids are P2P networks and clusters. From my understanding at this point they key is shared computing resources. Some basic definition and information can be found at:  http://en.wikipedia.org/wiki/Grid_computing. A question that came to mind when reading the introductory material is the difference between cloud computing and grid computing? A good response to this question can be found here: http://www.ibm.com/developerworks/web/library/wa-cloudgrid/. In essence, cloud computing goes a bit further than grid computing by dynamically allocating resources on demand.

example of a grid