22 November 2004

Results of simulated search engine failure

Summary

This discussion outlines the challenges of providing emergency communications from regions where there is limited access to the internet.

This note outlines a scenario whereby users need to provide information, yet the internet is either down or there are multiple barriers to entry.

Note discusses various failure modes warranting consideration of solutions.

This test was not coordinated with any agency or organization.

Discussion

Occasionally, internet access is denied. This can be scheduled downtime, or as a result of an international crisis.

We simulated a total internet failure and attempted workarounds to re-enter the communication channels. We assumed no human contact; no contact via TV, radio, landline, satellite, or audio-methods.

In short, we were isolated without any technical help, assistance or guidance.

The first step was to access the internet. We found that although the internet is sometimes blocked off, that it is not what it appears to be.

In some cases although access is denied and web pages do not appear, one can still access the internet. We refer to the bloggers in Iraq for additional patience and guidance on these methods.

In short, although major search engines may not be operable, take careful note that there are non-standard search engines available on other sites.

Of interest was the ability to access certain news organizations and their web-capable search engines in only select regions of the world. This is an opportunity for firms to advertise themselves as a back-up system should primary search engines fail.

Also, during this simulated disaster, we attempted to reconnect with the internet. At no time we we actually able to access our primary e-mail systems, nor were we able to re-establish contact with the existing e-mail providers.

In short, we assumed that all primary e-mail systems had been targeted, knocked out, and our job was to re-establish contact.

The first goal was to establish a new e-mail account; then attempt to provide a simulated warning-status message to blogger.

The results were troubling. It was difficult to find an e-mail account in such under such an isolated set of parameters. The e-mail account had to be confined to a geographic region.

Also, we found that certain search engines based on their geography could gain wide appeal in situations where a primary search engine failed in the United States, but non-US search engines were available.

The third problem was once we found this search engine was to be able to read the contents of the webpages. Again, although we had access to the isolated webpage in once country, this replacement-website provided hits back to websites we were not able to access.

Thus, it would be useful if there were known geographic-related factors affecting search engines, that those websites in the affected area be either flagged or removed as an option to expedite searching. In short, during a time of crisis, one wants to get access to usable web pages, and there is little concern for those that provide no content.

On the other hand, although the website itself is not accessible, there are times when the back-up search engine can still provide valuable information in just the small information provided in the lead.

One optimal solution would be for search engines to permit users to choose to expand or contract not only the listing, but the entire RSS-like feed of the websites. This would allow users to still read a website in the back-up web page, even when the webpage was not directly accessible.

The next problem was actually finding an e-mail account; then providing inputs back to blogger. Again, because there was no warning, we had no adequately established the blogger-codes necessary for direct input. This has been corrected.

However, one problem was finding the e-mail account. This was never accomplished in the 24-hour window.

Also, once messages were sent via a simulated fix, there were time delays, some messages lost, and in some cases we got a return message without useful information as to the nature of the error-mode, or what could be done to fix the problem.

This discussion goes into details of these error messages in the return messages and also provides a first cut at some insights of this simulated internet disaster.

Error Message

Your message could not be posted because of the following reason(s):
[Blogger e-mail removed], XML-RPC Error or Publishing Problem

Error discussion Ref

Solution Ref

Analysis

The link states that the reason for the error message is the sender is not using the e-mail address pre-loaded into the blogger e-mail preferences.

[Disclaimer: It remains to be understood whether this is the true reason. Indications are that this is not necessarily true; will attempt to send e-mails to this blog through a "non-matching e-mail origin".]

Let's presume the solution is valid

Until further testing is complete, let's presume the "cause for the error" [that the e-mail origin does not match what blogger already has loaded] is correct.

However, it is problematic to confine the "original e-mail" to that which is pre-loaded into the blogger-preferences for several reasons:

- Users may not be sending their e-mail-to-blog messages from a single source;

- Users may not be able to access the "pre-determined origin e-mail"

- Users may have a variety of other reasons making it impossible to send information from a pre-defined e-mail source.

The rule [that e-mail to blogger only works when the origin e-mail matches that which is pre-programmed into blogger] assumes [if true] that senders can only send the blog-entry from the e-mail you're registered with.

However, there is a problem: Suppose the e-mail sender has an expired account [that has been pre-programmed], but they do not realize it, or they are unable to access it, or they are attempting to send a message from a new e-mail-origin, but are unable to access blogger to update the information.

It is not correct to presume "if they can't get access to blogger, then they can't get access to an e-mail system". In some cases, google-blogger is down, not accessible, but there are non-google e-mail address that are still accessible.

Furthermore, in a crisis situation where e-mail sites are getting taken down by a worm, virus, or hacker attack, there may be situations where for security purposes blogger has been isolated/insulated, while other non-blogger sites are still accessible.

In a worst case situations, all sites, but a few may be taken down. Either text messaging may still be an option [despite the pre=programmed e-mail either being lost, scrambled, deleted, changed, or invalid].

In a test case, we simulated a massive failure of the internet, and assumed that a cyber attack had taken down all sites, except for a sole e-mail origin. The test case did not identify this origin-email center.

The goal of the exercise was to determine what methods could be used to identify whether the sites were down; what alternatives existed; and other methods to gain access to the e-mail blogger.

We share the results.

First, the simulation presupposed that the internet failed, provided no access, and there was no human contact or interaction to determine alternate sites or other modes to interact with other users, ISPs, or alternatives websites.

The challenges was to find another means to communicate, send messages, and identify the information needed to host information on the blog.

In a worst-case situation, there will be no instructions other than those that users can find on the internet, or that are already loaded. This exercise found several problems.

1. The ability to identify a new e-mail center

2. The problem with being able to reconfigure blogger to accept e-mails from a new e-mail origin;

3. The problem of being unable to send message without having pre-loaded information into blogger.

RECOMMENDATIONS

It would be nice if the following options were possible:

- In cases where the blogger system is still functioning as a host, but there is no means to access the system, that users can enter their blogger system through a pre-loaded code permitting an e-mail system-input to reconfigure blogger automatically. Clearly, this is problematic in that hackers can adjust the codes.

However, the goal is so that in the worst case situation whereby users are unable to directly access blogger and change codes, that there is a pre-listed and publicly available set of instructions that users can read while surfing the various search engines.

Ideally, the instructions should be short, use common phrases, and be sufficiently short so as to appear in the small space of a few lines on a non-standard search engine output. [Recall: The simulation assumed that google and major search engines were not accessible].

Other lessons

Blog entries went directly to desired content, but the covering URL when linked fail to go to the blog entry. In the worst case situation, the ideal option would be to have as a first entry both the covering URL and the date of the post, thereby facilitating searching and direct connection. It remains unclear why bonafide searches into blogger using alternative search engines could find material, but links to the actual blogs only took users to the covering URL.

Detailed scenario

Let us explore the potential problems associated with e-mail to blogger system. At this juncture, although we have yet to understand whether the direct linking

Scenario: You're in a place in the world where you've been denied access to all your existing accounts; you are not allowed to enter anything new, and the sites do not permit you to gain access.

All seems hopeless, until you suddenly realize you still can access a rather nebulous, unknown, and hidden website that happens to have a search engine linked to the web [that they have not blocked]...

Suddenly you have an idea: All you have to do is search for a site that has an e-mail forwarding system [for news articles], and you can post from the web [that is otherwise being blocked]....and then post this information into your blog.

This works, in theory, until you realize:

- You have not configured your blogger to do this;
- Or, blogger refuse to let you post the message because your sending e-mail doesn't match what is loaded into the computer

Situations where alternatives needed

Benefits of having a "any e-mail can access the system, and it doesn't matter where you are sending the information from" ... what if a user:

A. Forgets the e-mail
B. Cannot connect to that e-mail system
C. Wants to forward an article link directly to the blog
D. Has a new e-mail system, or has set up a back-up e-mail system.

Question for the day:

What's the point of having a "secret code" on the e-mail [of blogspot e-mail] if you also are restricted to the origin-email.

Philosophy

There's no reason to have a "destination only" and a "origin only" parameter in the blog spot.

Rant Extracted from upset field personnel

It's unfortunate that when the "stuff hits the fire", and you're being chased by [redacted: Undisclosed location, country, personnel], that you want to be able to quickly send a message, rather than wade through a bunch of convoluted codes.

At this end, have no time to sit around and wait for the system to "confirm" or "not confirm" -- I need something that's going to work right, post the information that I want in my blog, and let me get out of there without having to sit around; and I don't want to wait to "make sure" that my e-mail system that I'm ~sending~ from matches a ~required-pre-determined~ blog-spot-parameter.

In the field, I have no way of knowing "in advance" what the alternate e-mail system or e-mail that I might have to use, set-up, or send from as I don't know in advance that the specific site is going to crash when I most need it.

Summary

Google's e-mail to blogger option is outstanding. If only it would work in a timely manner.

It would be useful if there was some blogger knowledge of this subject, perhaps some discussion on alternatives, and some validation testing of other approaches and the failure modes.

In the meantime, we hope that blogger recognizes that users may not know the origin e-mail address, or there may be situations where a reconfirmation is required remotely in cases where users do not have conventional nor direct access to blogger.

The situations may seem extreme, but let's also remember where people are also finding themselves: In places that are remote, cut off from the world. These are not simply related to conflict and international crisis, but in some cases local security and internet testing requires sites and IPs to be shutdown without notice.

Perhaps this note might serve as a useful point of discussion on solutions, alternative methods to both present and discuss the issue.
Summary

This discussion outlines the challenges of providing emergency communications from regions where there is limited access to the internet.

This note outlines a scenario whereby users need to provide information, yet the internet is either down or there are multiple barriers to entry.

Note discusses various failure modes warranting consideration of solutions.

This test was not coordinated with any agency or organization.

Discussion

Occasionally, internet access is denied. This can be scheduled downtime, or as a result of an international crisis.

We simulated a total internet failure and attempted workarounds to re-enter the communication channels. We assumed no human contact; no contact via TV, radio, landline, satellite, or audio-methods.

In short, we were isolated without any technical help, assistance or guidance.

The first step was to access the internet. We found that although the internet is sometimes blocked off, that it is not what it appears to be.

In some cases although access is denied and web pages do not appear, one can still access the internet. We refer to the bloggers in Iraq for additional patience and guidance on these methods.

In short, although major search engines may not be operable, take careful note that there are non-standard search engines available on other sites.

Of interest was the ability to access certain news organizations and their web-capable search engines in only select regions of the world. This is an opportunity for firms to advertise themselves as a back-up system should primary search engines fail.

Also, during this simulated disaster, we attempted to reconnect with the internet. At no time we we actually able to access our primary e-mail systems, nor were we able to re-establish contact with the existing e-mail providers.

In short, we assumed that all primary e-mail systems had been targeted, knocked out, and our job was to re-establish contact.

The first goal was to establish a new e-mail account; then attempt to provide a simulated warning-status message to blogger.

The results were troubling. It was difficult to find an e-mail account in such under such an isolated set of parameters. The e-mail account had to be confined to a geographic region.

Also, we found that certain search engines based on their geography could gain wide appeal in situations where a primary search engine failed in the United States, but non-US search engines were available.

The third problem was once we found this search engine was to be able to read the contents of the webpages. Again, although we had access to the isolated webpage in once country, this replacement-website provided hits back to websites we were not able to access.

Thus, it would be useful if there were known geographic-related factors affecting search engines, that those websites in the affected area be either flagged or removed as an option to expedite searching. In short, during a time of crisis, one wants to get access to usable web pages, and there is little concern for those that provide no content.

On the other hand, although the website itself is not accessible, there are times when the back-up search engine can still provide valuable information in just the small information provided in the lead.

One optimal solution would be for search engines to permit users to choose to expand or contract not only the listing, but the entire RSS-like feed of the websites. This would allow users to still read a website in the back-up web page, even when the webpage was not directly accessible.

The next problem was actually finding an e-mail account; then providing inputs back to blogger. Again, because there was no warning, we had no adequately established the blogger-codes necessary for direct input. This has been corrected.

However, one problem was finding the e-mail account. This was never accomplished in the 24-hour window.

Also, once messages were sent via a simulated fix, there were time delays, some messages lost, and in some cases we got a return message without useful information as to the nature of the error-mode, or what could be done to fix the problem.

This discussion goes into details of these error messages in the return messages and also provides a first cut at some insights of this simulated internet disaster.

Error Message

Your message could not be posted because of the following reason(s):
[Blogger e-mail removed], XML-RPC Error or Publishing Problem

Error discussion Ref

Solution Ref

Analysis

The link states that the reason for the error message is the sender is not using the e-mail address pre-loaded into the blogger e-mail preferences.

[Disclaimer: It remains to be understood whether this is the true reason. Indications are that this is not necessarily true; will attempt to send e-mails to this blog through a "non-matching e-mail origin".]

Let's presume the solution is valid

Until further testing is complete, let's presume the "cause for the error" [that the e-mail origin does not match what blogger already has loaded] is correct.

However, it is problematic to confine the "original e-mail" to that which is pre-loaded into the blogger-preferences for several reasons:

- Users may not be sending their e-mail-to-blog messages from a single source;

- Users may not be able to access the "pre-determined origin e-mail"

- Users may have a variety of other reasons making it impossible to send information from a pre-defined e-mail source.

The rule [that e-mail to blogger only works when the origin e-mail matches that which is pre-programmed into blogger] assumes [if true] that senders can only send the blog-entry from the e-mail you're registered with.

However, there is a problem: Suppose the e-mail sender has an expired account [that has been pre-programmed], but they do not realize it, or they are unable to access it, or they are attempting to send a message from a new e-mail-origin, but are unable to access blogger to update the information.

It is not correct to presume "if they can't get access to blogger, then they can't get access to an e-mail system". In some cases, google-blogger is down, not accessible, but there are non-google e-mail address that are still accessible.

Furthermore, in a crisis situation where e-mail sites are getting taken down by a worm, virus, or hacker attack, there may be situations where for security purposes blogger has been isolated/insulated, while other non-blogger sites are still accessible.

In a worst case situations, all sites, but a few may be taken down. Either text messaging may still be an option [despite the pre=programmed e-mail either being lost, scrambled, deleted, changed, or invalid].

In a test case, we simulated a massive failure of the internet, and assumed that a cyber attack had taken down all sites, except for a sole e-mail origin. The test case did not identify this origin-email center.

The goal of the exercise was to determine what methods could be used to identify whether the sites were down; what alternatives existed; and other methods to gain access to the e-mail blogger.

We share the results.

First, the simulation presupposed that the internet failed, provided no access, and there was no human contact or interaction to determine alternate sites or other modes to interact with other users, ISPs, or alternatives websites.

The challenges was to find another means to communicate, send messages, and identify the information needed to host information on the blog.

In a worst-case situation, there will be no instructions other than those that users can find on the internet, or that are already loaded. This exercise found several problems.

1. The ability to identify a new e-mail center

2. The problem with being able to reconfigure blogger to accept e-mails from a new e-mail origin;

3. The problem of being unable to send message without having pre-loaded information into blogger.

RECOMMENDATIONS

It would be nice if the following options were possible:

- In cases where the blogger system is still functioning as a host, but there is no means to access the system, that users can enter their blogger system through a pre-loaded code permitting an e-mail system-input to reconfigure blogger automatically. Clearly, this is problematic in that hackers can adjust the codes.

However, the goal is so that in the worst case situation whereby users are unable to directly access blogger and change codes, that there is a pre-listed and publicly available set of instructions that users can read while surfing the various search engines.

Ideally, the instructions should be short, use common phrases, and be sufficiently short so as to appear in the small space of a few lines on a non-standard search engine output. [Recall: The simulation assumed that google and major search engines were not accessible].

Other lessons

Blog entries went directly to desired content, but the covering URL when linked fail to go to the blog entry. In the worst case situation, the ideal option would be to have as a first entry both the covering URL and the date of the post, thereby facilitating searching and direct connection. It remains unclear why bonafide searches into blogger using alternative search engines could find material, but links to the actual blogs only took users to the covering URL.

Detailed scenario

Let us explore the potential problems associated with e-mail to blogger system. At this juncture, although we have yet to understand whether the direct linking

Scenario: You're in a place in the world where you've been denied access to all your existing accounts; you are not allowed to enter anything new, and the sites do not permit you to gain access.

All seems hopeless, until you suddenly realize you still can access a rather nebulous, unknown, and hidden website that happens to have a search engine linked to the web [that they have not blocked]...

Suddenly you have an idea: All you have to do is search for a site that has an e-mail forwarding system [for news articles], and you can post from the web [that is otherwise being blocked]....and then post this information into your blog.

This works, in theory, until you realize:

- You have not configured your blogger to do this;
- Or, blogger refuse to let you post the message because your sending e-mail doesn't match what is loaded into the computer

Situations where alternatives needed

Benefits of having a "any e-mail can access the system, and it doesn't matter where you are sending the information from" ... what if a user:

A. Forgets the e-mail
B. Cannot connect to that e-mail system
C. Wants to forward an article link directly to the blog
D. Has a new e-mail system, or has set up a back-up e-mail system.

Question for the day:

What's the point of having a "secret code" on the e-mail [of blogspot e-mail] if you also are restricted to the origin-email.

Philosophy

There's no reason to have a "destination only" and a "origin only" parameter in the blog spot.

Rant Extracted from upset field personnel

It's unfortunate that when the "stuff hits the fire", and you're being chased by [redacted: Undisclosed location, country, personnel], that you want to be able to quickly send a message, rather than wade through a bunch of convoluted codes.

At this end, have no time to sit around and wait for the system to "confirm" or "not confirm" -- I need something that's going to work right, post the information that I want in my blog, and let me get out of there without having to sit around; and I don't want to wait to "make sure" that my e-mail system that I'm ~sending~ from matches a ~required-pre-determined~ blog-spot-parameter.

In the field, I have no way of knowing "in advance" what the alternate e-mail system or e-mail that I might have to use, set-up, or send from as I don't know in advance that the specific site is going to crash when I most need it.

Summary

Google's e-mail to blogger option is outstanding. If only it would work in a timely manner.

It would be useful if there was some blogger knowledge of this subject, perhaps some discussion on alternatives, and some validation testing of other approaches and the failure modes.

In the meantime, we hope that blogger recognizes that users may not know the origin e-mail address, or there may be situations where a reconfirmation is required remotely in cases where users do not have conventional nor direct access to blogger.

The situations may seem extreme, but let's also remember where people are also finding themselves: In places that are remote, cut off from the world. These are not simply related to conflict and international crisis, but in some cases local security and internet testing requires sites and IPs to be shutdown without notice.

Perhaps this note might serve as a useful point of discussion on solutions, alternative methods to both present and discuss the issue.
" />