Robots.txt generator including nasty bots blocker

Not all subdirectories were made for all people; at least not to see. For example, you may not want a Google cache exposing the mistakes of a test site you’re working on. Similarly, you may have a photo albumn that you’d rather not find itself showing up on search.Yahoo.com. How do you avoid various search engines indexing these directories while including the rest? Simple …

A long time ago, on a blog far away we discussed the ‘Robot Exclusion Tutorial‘ using just enough geek so you don’t shoot your foot clean off. Meaning, by simply using the robots exclusion standard, you can usually keep most ‘well behaved‘ search engines from ‘spidering’ into semi-private directories.

Note the emphasis on ‘well behaved.’ There are some nasty-bots out there that of course look at such entries as an engraved invitation to sneak a peak. Mark Pilgrim wrote about such nere-do-wells, even setting up a form of a ‘honey pot’ to ‘nail the suckas.’

That said, while searching for various Robots.txt validators, I came across a tool that generates a robots.txt file based upon entries you make … along with offering an option to include ‘nasty bots’ though I think from the prompts on the page, the webmaster needs to get two things straight.

First, you’re not really ‘blocking’ anything but requesting that a search engine not traverse up a stated path. Second, you don’t make an entry like ‘www.yoursite.com/private/ ‘ but rather ‘/private’.

Once you get past that the robots.txt generator tool works as advertised. Once you’re done with that, you may also want to visit my post listing posts on blocking sites using mod_rewrite via .htaccess and other such fun.

Too bad we just can’t send all the spammers on a rocket ship to the sun or something.

Posted in Uncategorized

UML diagramming for fun and prophet

Aim for nothing and you’ll surely hit it – and aphorism that is as true for software as it is life in general. So why do our church websites suck? In part because we don’t know to speak the language of software design. Below are some links to some simple tutorials for the object oriented modeling language known industry-wide as UML.
“Therefore I do not run like a man running aimlessly; I do not fight like a man beating the air.” – I Cor. 9:26

There is an old Baptist aphorism that goes “aim for nothing and you’ll surely hit it.” Not only is this true for one’s spiritual life, but it goes for software as well. I mean, imagine building a house without a blueprint? Yet that’s what we do all too often — in part because we don’t know to speak the language of software design.

Below are some links to some very simple tutorials that attempt to translate a very complex object oriented modeling methodology otherwise known as the Unified Modeling Language — or UML as it is known industry-wide.

Learning UML may be a bit of overkill for your church website, but my intent here isn’t for you to become an anal-retentive software analyst. Rather it is to get you thinking about diagramming your church websites design, layout and interaction. Specifically, I need you to think about the sequence of events, how users interact with your system and how data is stored and delivered.

Without doing this, at any level will lead to me visiting your website and picking it apart because you didn’t bother to measure twice before cutting … you’ve been warned:

Short and Sweet:
SmartDraw UML Center – How to Draw UML Diagrams in both the handy online HTML version, and the ‘print-it-later for the “reading-room” PDF’ flavor.

Extended Reading
Let’s say your rock-n-roll band has a long bus trip from Frogger, Virginia to Lizard Lick, North Carolina. Fear not! Developer.com has produced a nice sequential series of articles on the topic that are as fun to read as they are to eat. For your convenience, I’ve enumerated them below ordinally:

  1. UML Overview – What is it? Why do I need it? What about my needs?
  2. So you want to spend some money on some UML Tools? HA! Well th ink again, some are free (as beer).
  3. Creating Use Case Diagrams – Identifying the primary elements.
  4. The UML Class Diagram: Part 1 – Breaking down into its smallest parts.
  5. The UML Class Diagram: Part II – Practical examples.
  6. Object Diagrams in UML – Piecing the individual classes together into a single system.
  7. State Diagram in UML – Flagging when interactions happen, even when they don’t.
  8. Activity Diagram in UML – Let’s get busy people. Put them objects to work.
  9. Sequence Diagram in UML – If you read this, then ‘x’ will happen. Get it?
  10. Collaboration Diagram in UML – Can’t we all just get along?
  11. Component Diagrams in UML – Better programming through reusability.

Finally, massive mea-culprits for not writing sooner. The new job has kept me VERY busy. That and the fact that my wife and kid are still in MD getting the house packed up, it’s been a bit tough. Pray for me — and my broken foot.

Now go learn how to design stuff.

Posted in Uncategorized