Categories
Advanced Network Security

FIT5037 – Advanced Network Security Week 2

Miller-Rabin Python Implementation (slow but readable):

http://mchost/sourcecode/Miller-Rabin.py

As with my other subjects for week 2 I was absent for Adv. Network Security so this will be a summary of the lecture notes and reading materials. The title for this weeks lecture was ‘Adv. Cryptology, RSA and its implementation’. Considering the extensive assignment we completed last semester on PGP/GPG and it’s utilization of the RSA public key system, this will most likely be somewhat of a revision. I wrote a summary of the RSA system in that assignment which is will paraphrase below:

Generating Public and Private Keys (RSA):

Step 1: Generate two prime numbers
n = pq (let’s make p = 5 and q = 7)
5 * 7 = 35
n = 35
Step 2: Calculate the totient of n
φ(n) = (p – 1)(q – 1), φ is Euler's totient function
(5 - 1)(7 - 1)
φ(n) = 24
Step 3: Choose an integer, e, that is between 1 and φ(n) and co-prime with φ(n)
1 ,2 , 3 and 4 are not co-prime, however 5 is.
Let e = 5.
(e, n), (5, 35) is the public key.
Step 4: Using the public key and p*q (n), find the private key, d by finding the modular multiplicative inverse of e (mod(φ(n))
    d = e^–1 mod φ(n)
    d = 5^-1 mod φ(24)
Apply the Extended Euclidean algorithm (see http://mchost/sourcecode/eea.py)
    d = 29
public key = (5, 35)
private key = (29, 35)
 The encryption process for RSA is as follows:
    plaintext message = m, public key = (e, n)
    m^e mod(n) = cypher-text

The decryption process follows as:
    cypher-text message = c, private key = (d, n)
    c^d mod(n) = plaintext message

Signing of documents can be done, ideally using a hash function, a private key and a trusted certificate for the public key:
plaintext message = m, public key = (e, n), private key = (d, n)
hashFunction(m)^d mod(n) = signature
A recipient can confirm the signature with the following process:
    signature^e mod (n) = hashFunction(m)

The lecture notes explain these processes with much more correct mathematical notation, however this is the easiest way for me to express the process.

Also discussed in the lecture was a topic generating and tesing prime numbers. I did not complete strong analysis of ths process in the past semester. The Miller-Rabin test was introduced here. As per usual I find the easiest way to get my head around mathematical algorithms is not reviewing the mathematical proof/concept but by writing a script implementing the algorithm: http://mchost/sourcecode/Miller-Rabin.py

Categories
IT Research Methods

FIT5185 – IT Research Methods Week 2

Unfortunately I was absent for week 2 of IT Research Methods and the lecture delivered by Prof. David Arnott. The lecture was focussed on the initial stages to any research project, literature review.

  • Thematic Analysis – Qualitative in nature, classifying papers according to themes that are relevant to your research project.
  • Bibliographic Analysis – Quantitative in nature, using citation and/or content analysis. (rarely used in IT research)

A question posed at the start of the lecture; what is scientific evidence? Journal and conference papers along with websites, blogs, book and trade magazines were listed as possibilities. Before reading through the lecture I feel that any of these mediums could qualify as scientific evidence. Peer reviewed academics articles would however present a much more filtered source with blogs and websites most likely containing much more refutable contentions. It seems unwise to completely discount a source of information purely on the ground that it is a blog or website though.

The notes go on to present a rating system for journals, A, B and C, the A listers being:

  • Decision Support Systems
  • European Journal of Information Systems
  • Information and Management
  • Information Systems Journal
  • Information Systems Research
  • Journal of Information Technology
  • Journal of Management Information Systems
  • Journal of the Association for Information Systems
  • MIS Quarterly

The aim of a literature review can be summarized as:

  • Synthesis of articles
  • Define and understand relevant controversies
  • Based on critical review (note notes or observations)
  • Reads like an essay (but can use tables)

It seems that the thematic method of literature review is the avenue we will be encouraged to follow, which seems quite reasonable. Thematic review can be author and/or topic centric. Author centric review would only be appropriate in very limited niche topics where the published articles are by a limited number of researchers. When taking on topic centric review, creating a table with concept categorization for articles is recommended:

conceptMatrix
Webster & Watson Concept Matrix - Source week 2 lecture notes

Some questions are presented at the close of the lecture (which I imagine were answered in the lecture):

  • How long should a lit review be?
  • How many papers should be reviewed?
  • What tense should be used?
  • Which citation methodology? APA/Harvard?

I will have to follow up on these in the coming tutorial.

Finally there was a youtube video listed in the review materials for the week which included some good points:

  • What is the purpose of a literature review?
  1. Summarized what has been researched before
  2. Highlights the research gaps that you will aim to fill
  3. Why it is necessary to fill those gaps
  4. Set the scope of your research
  • Scope and length? – Does it need to be everything you know? No, the current state of the theory. Length requires discussion wit supervisor, but consider this is a summary of current research. Summary of existing knowledge, review of current research.
    Look for flaws, disagreement among researchers.
  • Sources – Refereed international journals, Books/Chapters, national journals, conference papers, non-refereed articles.
  • Review of instruments – What are you using to gather data to support your hypothesis, are they an acceptable source, why?

 

Basic Framework:

  1. Introduction
  2. Broader Communication Issues
  3. Likely Causes (Attack methods/motivations/scenarios)
  4. Mitigation Methods
  5. Summary of literature
  6. Research aims

Make a check list for evaluating articles!

Categories
Reading Unit - DoS Research

FIT5108 – DoS Reading Unit Part 2

This week’ summary will be a review of 2 papers from my reading pack: http://mchost/sourcecode/DoS/DoS%20Docs/JournalandBook/

Adaptive Defense Against Various Network Attacks, 2006, Zou, C., Duffield, N., Townsley, D., Gong W., IEEE.

Summary:

The method discussed in the paper was not focused on improving the current malicious packet identification methods but to increase the efficiency of their application by modifying their adjustable parameters based on the current and recent network conditions. One example was drawn via the Hop Count Filtering [HFC] method for mitigation of DDoS attacks through the assumption that attackers do not know the real hop-length from spoofed sources to their target. The effectiveness of this particular mitigation method is not paramount to the contention but rather the fact the HCF has adjustable parameters in its filtration. By adjusting the ‘strictness’ of the HCF using a simple, low overhead, low computational cost method, the authors were able to significantly improve the performance of the HCF.

Note that performance was based on a curve whereby the costs of false positive and false negatives were arbitrarily defined.

Relevance to thoughts in intelligent systems in network security and DDoS mitigation:

  • Computational cost is almost always relevant
  • Network overhead is always relevant
  • The utility, or cost/reward heuristic for the adaptive system must be provided to the system
  • Parameter management of multiple non-adaptive mitigation or defense systems can be done by single adaptive service which monitor network conditions and established the probability a current attack and the severity.
  • The proposed systems does not use any intelligent systems in the actual identification of malicious packets, perhaps this is due to the computational cost.
  • Adaptive systems can be used to achieve cost minimization for security services.

A Distributed Throttling Approach for Handling High Bandwidth Aggregate, 2007, Wei Tan, C., Chiu, D-M., Lui, J., Yau, D., IEEE.

Summary:

This article approach the breakdown of network communication in the case of flash crowds and DDoS attack which both cause high network aggregates sourced from distributed source to a single location. The authors propose what I would describe as a layered router throttling approach. Throughout the article the term ‘dropped traffic’ is used as to describe the effect of router throttling. The article provides some background on the router throttling strategy but I am somewhat confused over the dropping of traffic ones a certain bandwidth level is exceeded. Does this mean that all incoming packets will be dropped regardless of the existence of tcp session? Does it means that existing sessions will remain alive until they time out? I will need to do some further reading on the router throttling mitigation method. A key requirement of this strategy is having a number of routers in the preceding network hops subscribed to the method. See below:

throttlingapproach
Drawn from the paper, this is a deisrible router structure

The paper goes on to propose a number of algorithms and lightweight communication between routers and evidence that by dropping traffic the distributed throttling method can keep target servers alive. Although this solution would undeniably be effective in keeping a server alive, it drops traffic based on the traffic level of the router it approaches. I feel that there is a very bad worst case scenario where the probability of packet being dropped would have a very low correlation to whether or not it is malicious. The lack of header/packet inspection does have very good computational efficiency however.

  • This solution could be consider somewhat of a benchmark that intelligent mitigation methods would need to improve on, the indiscriminate dropping of packets will result in DoS for users approaching the server via routes that the DDoS approach. Keeping a server alive is probably the primary goal of DoS mitigation but service availability should stand right next to that goal.
  • If this defense strategy was widespread it appears to have numerour vulnerabilities that attackers would sureley exploit. Attackers could test thrasholds for tripping packet dropping and possibly launch attacks that deny serverhere is a  to specific regions with less cost than an attack on a service that was not protected by distributed router throttling.
  • I get a sense that this strategy could work on a macro sense perhaps piggy backing on border routing protocols, however the ‘dumb’ nature of throttling seems a very limiting factor. I will obvously need to investigate the router throttling methods more as with my current understanding this solution seems sub-optimal.

 

 

Categories
Advanced Network Security

FIT5037 – Advanced Network Security Week 1

Week 1 of Adv network security to be lectured by Dr Phu Dung Le provided an introduction to the topics covered in the unit:

  • Modern computing and network security
  • Ellicptic curve public key encryption
  • Design and implementation of RSA and ECC
  • Intrusion detection systems
  • Network and distributed software security
  • Advance wireless security
  • Large computer security systems
  • Security, load balancing and network performance
  • Main research in security

The lecture broke off in to some very interesting discussion over information retrieval from encrypted data sources. The example provided seems like a one of case but this problem will become increasing relevant with the rise of cloud computing.  For example, as large companies such as Sony find strong efficiency and financial motivators to outsource their data storage to cloud providers, encryption of that data is paramount. With a large, off site, encrypted data sources there are issue with the efficient retrieval of data and the point of decryption. For example:

  • If searching for similar images given and initial image, how can this be accomplished without downloading and decrypting the entire database?
  • When retrieving data, at what point does decryption occur, if at the client then all the incoming data will fly straight past firewall, intrusion detection systems and anti-virus software.

A paper proposing a solution where:

an encryption scheme where each authorised user in the system has his own keys to encrypt and decrypt data. The scheme supports keyword search which enables the server to return only the encrypted data that satisfies an encrypted query without decrypting it.

http://mchost/sourcecode/papers/Sharedandsearchableencrypteddataforuntrustedservers.pdf

The problem of like image recognition is still not easily addressable using this solution. Although it could be argued that categorization schema could work effectively. I wonder at plausibility of using unsupervised neural networks in conjunction with the hash algorithm to provide a method not dependent on designer imposed categorization. Imagine the network would need to be infinitely complex to follow hashing however…

The tutorial introduced Snort (a leading intrusion detection system) – http://www.snort.org/

Installing and making a basic configuration for snort was the task.  I am not a big fan of the red hat linux distro that we have access to in the tutorials so I complete the install of snort 2.9.0.5 along with snort report 1.3.1 on my home gateway. I used the latest dynamic rules from

The tutorial I followed loosely for the install can be view:  http://www.symmetrixtech.com/articles/001-snortinstallguide.pdf (*note that following the instructions blindly will result in disaster).

It was also mentioned in the lecture that we would be investigating the RSA in comparison to Elliptic curve cryptology [ECC]. I had no idea what ECC was, a good video I found providing a brief explanation:

Categories
IT Research Methods

FIT5185 – IT Research Methods Week 1

Week 1 of IT research methods was a lecture by Dr Jose Kuzic on the nature of research.  The lecture bounced between subjective opinions from experience in research and a a framework for conducting research questions.

  • Formulating Questions
  • Literature Analysis
  • Case Studies
  • Surveys
  • Qualitative data analysis
  • Quantitative data analysis
  • Communication research

Also introduced were some research paradigms:

  • Scientific research (positivist)
  • Applied research (practical)
  • Social research (interpretive)

I feel that being aware of these paradigms is valuable but self imposing mutual exclusivity or black and white generalization would be counter productive (ie: oh well that’s just a positivist view/ I can’t do that I am doing applied research). A more pragmatic approach of using whatever the best method for reaching outcomes to a posed question regardless of paradigm would be required for good research.

inductiveDeduction
Induction and deduction in science (source: week 1 lecture notes)

Details of Assignment 1 and 2 were also made available on moodle this week. Assignment 1, a literature review and presentation seems like it will be an enjoyable assignment that will allow some synergy with other subjects.

 

Categories
Reading Unit - DoS Research

FIT5108 – DoS Reading Unit Part 1

I am undertaking a reading unit this semester focused on Denial of Service [DoS] attacks and their mitigation. As there are no subjects dedicated to this field a reading unit was the best option. The aims of the unit will be:

  1. Study system vulnerabilities and existing DoS attacks
  2. Propose a new method to mitigate one of the DoS attacks

I have not investigated DoS attacks on anything other than an introductory level prior to this so my blog notes will start from that point. With this in mind the best beginning is in definitions. Most of this introductory post will glean resources from wikipedia’s DoS page http://en.wikipedia.org/wiki/Denial-of-service_attack, see their reference list for further reading.

Denial of Service Attack:  To slow network performance or unavailability of services (web services). Issues can spread to network branches surrounding the targeted system. In some cases entire geographical regions can be prevented from accessing the external network.

DoS attacks can also be characterized where and attacker explicitly attempts to prevent legitimate users from accessing specific services. There are two major classifications:

  • Attacks which crash a server
  • Attacks which flood a server
DoS_Attack
Stachledraht DDoS attack, source: Wikipedia

There are five categories that DoS attacks can be placed:

  1. Consumption of computation resources (ie: HTTP-GET DDoS flood attack, http://teamxpc.com/forum/topic/155918-http-get-dos-attack-paper/)
  2. Disruption of configuration information  (ie: DNS Poisoning attack, http://www.spamstopshere.com/blog/2008/08/07/recent-dns-poisoning-exploit-used-for-dos-attacks/)
  3. Disruption of state information (ie: Resetting of TCP Sessions, http://kerneltrap.org/node/3072 , http://en.wikipedia.org/wiki/TCP_reset_attack)
  4. Disruption of physical network components (ie: physical access to servers, phlashing attack/PDoS, http://hackaday.com/2008/05/20/phlashing-denial-of-service-attack-the-new-hype/)
  5. Obstructing communication media (ie: replay attacks on wifi, http://www.aircrack-ng.org/doku.php?id=simple_wep_crack&DokuWiki=9a77f3d58e7c5e4adc840b60b1a2197e, cable cuts, http://www.guardian.co.uk/world/2011/apr/06/georgian-woman-cuts-web-access)

Some examples of known DoS attacks:

 

Some additional reading on DoS attack definitions:

http://www.garage4hackers.com/showthread.php?251-DOS-Attacks

Categories
Network security

FIT5044 – Network Security Week 11 + Review

The final week of new topics for Network Security covered Security for Large Computer Systems. This post will also contain a short review of the FIT5044 subject. The first point to consider when implementing large scale security solutions is the fast moving nature of computer security in addition to the difficulty in change associated with large business networks. Increased integration, particularly with  the availability of inter-organization or publicly available services adds difficulty considering the mutually exclusive nature of security and convenience.

Network Security topics for large organizations
source: Week 11 Lecture notes FIT5044

As can be seen there are a large number of areas were security must be actively enforce on a large network.

IDS systems to investigate:

Snort (http://www.snort.org)

Cisco IDS (http://www.cisco.com/warp/public/cc/pd/sqsw/sqidsz/index.shtml)

Subject Review:

FIT5044Network Security was my favorite subject of the MIT course thus far. It contains very interesting subject material and introduces students to topics they must independently investigate to gain proper understanding (I think all post graduate subject should subscribe to this). It is proposed that the subject should be a good addition for non-IT students however I imagine this would be quite challenging without some fundamental IT background. I recommend this subject to anyone in the MIT course.

Categories
Natural computation for intell. sys.

FIT5167 – Natural Computation Week 11 + Review

Let post as I forgot to publish,  the last week of new topics in Natural computation covered Recurrent networks for time series forecasting. The alternatives for structuring and feeding back previous time series are the main points of difference between methodologies.

Elman Networks:

elman network
source: Week 11 lecture notes FIT5167

Jordan Networks:

jordan networks
source: Week 11 lecture notes FIT5167

Fully recurrent:

Fully Recurrent Time series forcasting network
source: Week 11 Lecture notes FIT5167

These network operate very similarly to standard MultiLayer perceptrons. Self organizing maps have been proposed as one possible method for selecting input variables. Genetic algorithms were also noted as an alternative input selector.

Review of this unit:

I found the FIT5167 to be a very thought provoking subject, with excellent resource provided by the subject lecturer, Grace Rumantir. The best part of the subject was the assignments where we got some very useful practical experience  constructing neural networks. With the statistical analysis that NNs allow, the skills learned in the subject can be applied to a very wide range of problems. I would recommend this subject to anyone studying MIT at Monash even if their major is not Intelligent Systems.

Categories
Intelligent systems

FIT5047 – Intelligent Systems Week 11

Week 11 moved into recommender systems, perhaps one of the most popular and commonly used forms of AI. Sites such as Google and Amazon built their success on the effectiveness of their recommender systems (now I guess their brands can carry them for a while). The first topic of the lecture was association mining, given a large dataset, how do we find useful associations between attributes.

Support and confidence were proposed as useful metrics to drive this process. Unfortunately we found some conflicting definitions among Data Mining, Weka and R&N texts. When in doubt check wikipedia..:

The support supp(X) of an itemset X is defined as the proportion of transactions in the data set which contain the itemset.

supp(Z) = P(Z)

The confidence of a rule is defined – confidence

The lift of a rule is defined as – lift

The leverage of a rule is defined as –

leverage(X -> Y) = P(X and Y) – (P(X)P(Y))

The source listed in wikipedia for these definitions is:  http://michael.hahsler.net/research/association_rules/measures.html

In our lecture notes we had support for rule a -> b as the union of A and B, this confused me as I still think that support is the intersection of A and B.

The rules described above are quite intuitive when work through in an example. Lift feel like and extension of confidence taking into account independence. An increase in lift implies dependence.

No leverage implies independence between attributes and vice versa.

This topic was closed with the conclusion that it is in fact bad practice as variance and standard deviation are completely ignored.

A quick review of collaborative and content based filtering were covered next. Content Based Filtering [CBF] (haha) can be implemented using an array of machine learning  techniques already covered. Naive Bayes, Neural Networks and Decision trees are classification methods that can be applied to CBF. The pre-processing involved with CBF seems to be the most limiting factor. Term frequency and Inverse Document term frequency can be compile into tables allowing for effective searching. Considering the vast size of the data sets that these systems would be applied to, this can seem a bit daunting.

Collaborative filtering [CF] seems a bit easier to implement but does then rely on user participation. The introduction in the lecture felt very similar to the basics of Self Organising Maps. Vectors are created to represent instances (in  this case users). Euclidean distances (or some spin-off of this) are used to measure instance ‘likeness’ then missing values for instance vectors can be predicted based on the instances that are considered ‘like’. There was quite a bit of mathematical methodology described on the lecture slides which would be required when implementing a CF system.

Collaborative Filtering v Content Based Filtering
Collaborative Filtering v Content Based Filtering (soure: Week 11 lecture notes)
leverage(X -> Y) = P(X and Y) – (P(X)P(Y))
Categories
Intelligent systems

FIT5047 – Intelligent Systems Week 10

Week 10 moved on from classification to clustering. Although, conceptually, there was close relation to topics covered in Natural Computation the methods discussed were new. Again, Euclidean distance is a fundamental measure of similarity/uniquness.

The first method introduced was Heirarchical Clustering. This introduction was very bried and reference to the text would need to be made for issues such as linkages.

The next method was K-Means clustering.

 

kmeans
As cabn be seen with K = 3 we can move the center but the number of clusters is static

 

 

I find the limitation of assuming the number of clusters [K] to go close to invalidating this methodology in its basic form. Of course, however the algorithm can be extended to an exhaustive or stoichastic search were multiple K values are compared and contrasted. The idea of clustering is to simplify data sets, in essence reducing dimensianality. With this in mind there must be a penalty for extended K-means algorithms for the number of clusters. Otherwise the best clustering would always result in K = number of unique instances. MML, MDL and BIC are examples of algoriths that incorporate these penalities. Interestingly, I came across MDL when looking for effective method for discretizing continuous variables. It now seems obvious that discretization is a form of clustering where there need to be penalties for an increasing number of clusters. For more info on using MDL to discretize continuos variables see:

Fayyad, U., Irani, K., 1993, Multi-interval discretization of continuous valued attributes for
classification learning, Thirteenth International Joint Conference on Articial Intelligence, 1022-
1027

Interstingly Usama Fayyad is now Chief Data Officer and Executive Vice President, Yahoo! Inc… for next time anyone says research in this field is pointless for a career.

The lecture continued to introduce issues and algorithms which require a great deal of reading and writing to do justice (which I am yet to complete).

Chief Data Officer and Executive Vice President, Yahoo! Inc.