Step-By-Step Google App Engine Deployment... documentation?

[chrisjacob]

Hi Thatcher,

I'm really interested in your project. 1. because I'm about to embark on a fairly large jQuery AJAX app and 2. because I'm interested in how you have Claypool hosted on GAE.

It would be great if you could provide a guide or some tips on how to get hosted on GAE.

There is a lot of buzz right now regarding Google's Specification for Making AJAX Applications Crawlable. http://code.google.com/web/ajaxcrawling/docs/specification.html

I'm hoping to build a basic service on GAE that I can post a URL to and have it return a HTML Snapshot after all JS has been executed.

I've posted two questions on Stackoverflow... 1 for HTMLUnit on GAE and one for Watir on Heroku... still no concrete answers. Env.js + Claypool might just be the solution I'm looking for :-)... Also I imagine many people will be looking for "How the F*CK do I setup a headless browser!?" ... if your project can answer this I'm sure it will gather a lot of attention.

http://stackoverflow.com/questions/3517944/making-ajax-applications-crawlable-how-to-build-a-simple-web-service-on-google-a

http://stackoverflow.com/questions/3597118/can-you-deploy-watir-on-heroku-to-generate-html-snapshots-if-so-how

p.s. Congrats on your second child ;-)

[thatcher]

Awesome Chris, Thanks for the link to that, I hadnt seen it yet but just just read quickly and it very cool. Currently Claypool is designed to promote a similar approach mentioned in the same article with this link http://googlewebmastercentral.blogspot.com/2007/11/spiders-view-of-web-20.html (hijax basically means we use traditional links and urls but intercept the click events, form submissions etc and consume the data at the url as json, though the link itself is a valid html page). It very similar to the hash approach but has its own strengths and weaknesses.

I've been wanting to add a built-in "hash router" so working from the google spec would probably be a great place to start.

I will do write up for you today on how to deploy yo google apps engine, its seems easy to me, but I forget that not everyone uses claypool all the time ;) Running claypool on appengine with Envjs and jquery is a pretty fun combination and gives you a lot of options in how you tackle a given requirement.

Thanks again for the interest, I should be able to write something up at lunch time or early this evening. Also we are currently at a minor version increment for 1.2 beta, so it's a great to get in feature requests. Having a built in hash router that implements googles spec would be an awesome addition to 1.2.

Thatcher

[chrisjacob]

Hi Thatcher,

Thank you for such a quick reply. I'm looking forward to the "Deploy on GAE guide" :-).

I've been doing some thinking lately about Google's spec... it has some flaws right now:

1. Googlebot only - not yet on Yahoo, Bing, Ask, etc
2. All links have to be updated to #!
3. Hijax links were clickable for non-js users - if these are replaced with #! links then screen readers are screwed).

I read a great article on some of the flaws with Google's proposal but can't seem to find it now :(

I've been spec'ing out an idea for "Headless Hijax".
Essentially all the same Hijax implementation applies.... i.e. provide static alternatives to dynamic content. However, when you are not able to (or simply don't want to duplicate code into the backend to generate static pages) you have a "catchall" route that will return a HTML Snapshot for Search Engine bots (or non-js humans if they 'force' the HTML Snapshot - via a link displayed in a NOSCRIPT section).

You could have links like:

a href="/controller/action/id?key1=value1&key2=value2" onclick=”navigate('#controller&action&id&key1=value1&key2=value2'); return false” foo /a

... it would require some trickery, but the idea is that google would index pretty urls like "example.com/controller/action/id" and the backend would do a 301 redirect to "example.com/#controller&action&id" if the person is human... if it's a crawler they go to "example.com/controller/action/id" which returns a HTML Snapshot.

There are still flaws in this method of course ... but it's got some potential.

... my explanation is a bit "complex" right now - I'll be working on a very simple demo and then a more complex implementation - but swamped with work right now :(.

[chrisjacob]

href="/controller/action/id?key1=value1&key2=value2" onclick="#controller&action&id&key1=value1&key2=value2"

[thatcher]

Hey Chris, I wasnt able to get to the write up last night, but will try again today. A couple notes that might get you started with how Claypool provides hijax.

First you might want to check out the project http://rhino-for-webapps.appspot.com/, its also in my github repo, but by itself is enough to run jquery and envjs in jetty or tomcat or GAE. It also has some basic deployment directions.

Next you might want to look at recordsofexistence.com (its a small site I built for a friend but its my sandbox for building a hijax site). Basically the architecture work that every url exposes itself as rendered html or as just the json that is used to render the template. just add fo=json to any url for example:

http://www.recordsofexistence.com?fo=json
http://www.recordsofexistence.com/artists?fo=json
etc

Note also that the templates are jquery-tmpl so they can be rendered on the client or on the server. Claypool allows you to hijax all the links and forms based on the url patterns and all the controller needs to do is get the json and the template and render it on the client.

It's very close to what your talking about but doesnt rely on hash values at all, and requires not onclick or similar attributes actually in the html.

Lets keep this thread going because as I do this write up I'm going to go ahead and finish up recordsofexistence.com and make it a crawlable ajax app. If you feel like it, we can work through it together and it will give you a very clear understanding of how Claypool was built for exactly this issue.

Thatcher

[chrisjacob]

Hi Thatcher,

Rhino For Werbapps looks brilliant! Thanks for the tip off. I'll have a play with it as soon as possible to try building the HTML Snapshot service I mentioned.

Very keen to keep this thread going - and I'll look deeper into the recordsofexistence.com example provided. Interested in the technique you're using for sure :-)

I have to get back to work :-( damn it!

[chrisjacob]

Just realised how close I am to getting a HTML Snapshot service after seeing this example...

http://github.com/thatcher/rhino-for-webapps/blob/master/WEB-INF/examples/envjs/dispatcher.js

:-)