Billy Hoffman, an engineer at Atlanta company SPI Dynamics unveiled a new, smarter web-crawling application that behaves like a person using a browser, rather than a computer program. "Basically this nullifies any traditional form of forensics," says Hoffman.
Tim Ball, director of systems and development for the U.S. Senate's Democratic Policy Committee knows what it's like to be under constant spider attack. The Senate website relies extensively on server logs for forensics, but Ball is no longer confident that approach will be helpful in the long run.
Ball says the research will make it easier for attackers to automatically and discreetly spot flaws on websites they previously had to root out by hand. "What Billy's done is massively simplified the process and make it faster," says Ball.
Hoffman hopes the street will find its own positive uses for his work as well. "One of the really cool things I have had to do was to score how interesting a link would be," he says. His technique is similar to applications like Google's page scoring system, but is publicly available in open-source Java code anyone can use.
Much fun was had and much work was done by the entire crew in DC this week.
