Wednesday, November 14, 2007

Migrating to Seaside 2.8

I've been developing my web app with Squeak 3.9 and Seaside 2.7. This month, Seaside 2.8 was released and it promised faster rendering speed and lower memory consumption. So I decided to migrate my application over to it. But it was not without incident...

It turns out that in order for my web app to function properly using Apache2 and SSL, Seaside 2.8 requires the Server Port and Protocol for my app to be set to 443 and https, respectively. It's curious that this was not necessary under Seaside 2.7.

For some reason, I could not get my app to work initially with the Server Port and Protocol changes. After some tinkering, I managed to eventually get it working, but then it broke Apache2 load balancing! This became very frustrating.

So I decided to start from scratch and very carefully work my way to the finishing line. Methodically and meticulously, I built the Seaside images for my app (both 2.7 and 2.8 versions) and then small step by small step I configured Apache2 to run the apps. I was looking for clues as to what was wrong.

Well, to my pleasant surprise, everything worked! There was no problem! The lesson here is that one needs to be very, very careful in how one constructs the server environment for running a Seaside app with Apache2. One of the things I was particularly cautious of was to not do *anything* with Seaside's built-in WAKom web server--I left it in its default configuration. Under Linux, I was going to start the Seaside app by explicitly specifying the port number as in

squeak MyApp.image "" port 9090 &

and I was hoping that Apache could live with it. This may have been my best decision.

Squeak and SMTPClient

Apparently, no one uses SMTPClient because there is a bug that has lingered for ages. Using it raises the following Squeak error message:

TelnetProtocolError: 501 malformed auth input (#5.5.4)

Here's my test Squeak method:

mailTo: userAddress subject: subject text: text
....| client |
....client := SMTPClient openOnHostNamed: 'host_mail_server' port: 25.
....client user: 'my_user_id'; password: 'my_password'; login; initiateSession.
....client mailFrom: 'my_email_address';
........recipient: userAddress;
........data: text.
....client quit.


I managed to find a fix. In the #encodeString: method of the #SMTPClient in the Network-Protocols package, amend the last statement to read:

....^ str truncateTo: (str size - 2)

Perhaps it will be fixed in the next Squeak release. It's probably very low priority as I seem to be the only person who cares about it.

Saturday, November 10, 2007

A Divergence from Open Source

I'm ashamed to admit it, but the one area where Open Source has defeated me is in the area of video processing. I was looking for a means to convert MPEG-2 videos that I had captured from my Hauppauge (pronounced HOP-hog) WinTV-PVR-USB2 to MPEG-4 for serving to my clients. (This is the reason for Darwin Streaming Server, by the way.) The best Open Source software for video conversion apparently is FFmpeg, but this thing is so darn complicated--and no one has bothered to write a user-friendly GUI front-end for it--that I simply couldn't figure out how to use it. And as is typical of most Open Source projects, FFmpeg doesn't have any friendly documentation for newbies. Yes, there *is* detailed documentation, but it reads like a Grumman F-14 operational manual.

I was also looking for a way to add a digital watermark to these videos. There are very few products on the market that do this. And zero products from the Open Source community.

Fortunately, in the Windows world (blech!), there is a solution...sort of. It costs US$70 and it comes from Deskshare. It's called Video Edit Magic. This program is pretty straightforward to use and it has all kinds of nice features and options. It's actually quite versatile. Video Edit Magic should be a bargain at $70. The only problem is, Deskshare technical support is crappy. They rarely respond to my email requests for help and they don't provide telephone support (so you can't really pester them). When you submit an email request, they claim that they'll get back to you within one business day, but they always take longer than that if they even deign to respond. (I've submitted at least half a dozen requests over the past couple of months.)

What kinds of problems do I have with this software? Well, for starters, it won't run under Windows Vista, despite their claim. I initially purchased the program for a home-built Vista box (Antec Sonata II case, 2.4GHz Core 2 Duo, 2GB DDR2, 500GB hard drive). Video Edit Magic would occasionally crash for no apparent reason.

On a second machine that I subsequently bought from Dell (Dimension 9200, Core 2 Quad-Core, 3GB DDR2, 500GB hard drive), Video Edit Magic won't let me open any files! Whenever I click on the Open File menu or icon, the program would immediately close!

(On my aging Windows XP laptop, Video Edit Magic works just fine. So this is clearly a Vista issue.)

Video Edit Magic also has a bug with the digital watermarking feature: it won't let me alter the opacity of the watermark, even though there is a slider to vary the opacity (which is permanently stuck at 50%). Deskshare says they are aware of the bug but they can't give me a timeline on when it'll be fixed. So what am I supposed to do in the meantime??? I have important work to do!

My project is Open Source on the server side, but because of video processing requirements (Hauppauge support, Video Edit Magic) it has to be Windows on the client side. The only thing I can do is wait for Deskshare to come out with a new release sometime in the next couple of months. (Historically, their release schedule is once every four or five months. But I understand their next release represents a significant rewrite of the code base. This doesn't bode well.)

Friday, November 9, 2007

How to Install Squeak

Download the Squeak Virtual Machine for Linux from http://squeak.org. After you untar it, rename i686-pc-linux-gnu to i686-pc-linux-gnulibc1 before installing the Squeak VM.

sudo ./INSTALL

Alternatively, you can use apt-get to install Squeak:

http://wiki.squeak.org/squeak/3616

To use Apache to serve files for the Seaside app, Seaside's own internal web server is bypassed. So the files that Seaside normally serves from its internal libraries need to be extracted into Apache's file serving location. Here's what I discovered:

http://permalink.gmane.org/gmane.comp.lang.smalltalk.squeak.seaside/12549

To start your Seaside application for a given port number, say, 9090:

cd seaside_app_location
squeak MyApp.image "" port 9090 &

Thursday, November 8, 2007

How to Install Apache2 for SSL and Load Balancing, Part 3

If I add the following to my Apache config (inside the SSL virtual host):

<location /balancer-manager>
SetHandler balancer-manager
</location>

I can simply type in https://www.mydomain.com/balancer-manager and bring up a web-based configuration manager for Apache load balancing! Quite nifty!

The balance manger should be secured:

sudo mkdir /var/www/ssl/balancer-manager
sudo htpasswd -c /var/.htpasswd admin

sudo gedit /var/www/ssl/balancer-manager/.htaccess
and add:

AuthType Basic
AuthName "Members Only"
AuthUserFile /var/.htpasswd
<limit GET PUT POST>
require valid-user
</limit>

How to Install Apache2 for SSL and Load Balancing, Part 2

Here's my updated httpd.conf file for supporting load balancing. Note the key changes in green...

LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so
LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so
LoadModule ssl_module /usr/lib/apache2/modules/mod_ssl.so
LoadModule proxy_balancer_module /usr/lib/apache2/modules/mod_proxy_balancer.so

ServerName www.mydomain.com
NameVirtualHost 192.168.1.101:80

<virtualhost 192.168.1.101:80>
ServerName www.mydomain.com
ServerAlias mydomain.com
RewriteEngine on
ProxyRequests off
DocumentRoot /var/www
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ https://%{SERVER_NAME}/$1 [L,R]
</virtualhost>

NameVirtualHost 192.168.1.101:443
<virtualhost 192.168.1.101:443>
ServerName www.mydomain.com
RewriteEngine on
ProxyRequests off
ProxyPreserveHost on
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/apache.pem
DocumentRoot /var/www/ssl
ProxyPass /seaside/go balancer://seaside_cluster stickysession=server nofailover=on
ProxyPassReverse /seaside/go balancer://seaside_cluster
<proxy balancer://seaside_cluster>
BalancerMember http://localhost:9090/seaside/go route=MyApp9090
BalancerMember http://localhost:9091/seaside/go route=MyApp9091
BalancerMember http://localhost:9092/seaside/go route=MyApp9092
</proxy>
RewriteRule ^/$ balancer://seaside_cluster [P,L]
</virtualhost>


My Seaside app requires "sticky" sessions in the load balancer. Session affinity is done using cookies. The 'server' in stickysession=server refers to the identity of a cookie that must be named something like <some_arbitrary_string>.<route_identifier>. The route identifier comes from the route=MyApp<port number> above. (It's quite arbitrary--I could've used A_, B_, C_, etc.)

Your backend application server (in my case, the Seaside app) must also be configured to create this cookie (named 'server' and containing, for example, 'seaside.MyApp9090'). Here's a snippet of Seaside code to illustrate (the leading dots signify spaces):

initialRequest: aRequest
.... port
....self initialRequest: aRequest.
....port := ((HttpService allInstances
........select: [:each each isRunning])
............collect:[:each each portNumber]) first.
....self session currentRequest cookies at: #server
........ifAbsent: [self session redirectWithCookie:
............(WACookie
................key: #server
................value: 'seaside.',
....................((SmalltalkImage current imageName
........................copyAfterLast: FileDirectory slash asCharacter)
............................copyUpToLast: $.), port asString)]


The changes in red are for eliminating warning messages in Apache's error.log.

How to Install Apache2 for SSL and Load Balancing

sudo apt-get install apache2
I chose the default file serving location and changed its ownership for my convenience...
sudo chown richard:richard /var/www

Ubuntu (both Feisty Fawn and Gutsy Gibbon) is missing the apache2-ssl-certificate script. You can find it here (the download is called Apache2-ssl.tar.gz):

http://www.lingams.net/

Do this after you've extracted the files:

sudo mv ssleay.cnf /usr/share/apache2
sudo mv apache2-ssl-certificate /usr/sbin
sudo mkdir /etc/apache2/ssl


Don't do this (because a2enmod and a2ensite really screws things up!):

https://help.ubuntu.com/community/forum/server/apache2/SSL

(But it could be useful for other websites.)

Do this:

sudo apache2-ssl-certificate -days 365
sudo echo "Listen 443" | sudo tee -a /etc/apache2/ports.conf

Add the following to /etc/apache2/httpd.conf (substituting the appropriate IP address, server name, application name, etc.):

LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so
LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so
LoadModule ssl_module /usr/lib/apache2/modules/mod_ssl.so

ServerName www.mydomain.com
NameVirtualHost 192.168.1.101:80
<virtualhost 192.168.1.101:80>
RewriteEngine on
ProxyRequests off
DocumentRoot /var/www
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ https://%{SERVER_NAME}/$1 [L,R]
</virtualhost>

NameVirtualHost 192.168.1.101:443
<virtualhost 192.168.1.101:443>
RewriteEngine on
ProxyRequests off
ProxyPreserveHost on
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/apache.pem
DocumentRoot /var/www/ssl
ProxyPass /seaside/go http://localhost:9090/seaside/go
ProxyPassReverse /seaside/go http://localhost:9090/seaside/go
RewriteRule ^/$ http://localhost:9090/seaside/go/$1 [P,L]
</virtualhost>


I chose to put all my SSL-specific stuff in the ssl folder:
mkdir /var/www/ssl

sudo /etc/init.d/apache2 restart

This is very useful for load balancing:

http://www.howtoforge.com/load_balancing_apache_mod_proxy_balancer

Wednesday, November 7, 2007

How to Install PostgreSQL

http://www.supriyadisw.net/2007/02/postgresql-on-ubuntu-linux

Or just do this:

sudo apt-get install postgresql-8.2
sudo -u postgres psql template1
template1=# ALTER USER postgres WITH PASSWORD 'secret';
template1=# \q
sudo -u postgres createdb mydatabase
sudo gedit /etc/postgresql/8.2/main/pg_hba.conf
- I chose METHOD 'password' for IPv4 local connections

sudo /etc/init.d/postgresql-8.2 restart

How to Install Darwin Streaming Server

First, download the Darwin Streaming Server binary for Linux (yes, it says it's for Fedora Core 4 and later, but it'll work for Ubuntu, too). My download was named DarwinStreamingSrvr5.5.5-Linux.tar.gz.

If you can read German, the following site should be very useful:

http://wiki.ubuntuusers.de/Darwin_Streaming_Server

Or just do the following after you've extracted the files:

cd DarwinStreamingSrvrlinux-Linux
sudo addgroup --system qtss
sudo adduser --system --no-create-home --ingroup qtss qtss
sudo ./Install

To start the server:

cd DarwinStreamingSrvrlinux-Linux
sudo ./streamingadminserver.pl

You administer the streaming server through port 1220 in your web browser.

How to Install VNC (and SSH for secure connection)

First, you want to install SSH in order to ensure a secure connection:

sudo apt-get install ssh
System->Preferences->Remote Desktop
  • allow other users to view this desktop
  • don't ask for confirmation
  • require the user to enter a password

Now you install VNC. The following contains complete instructions:

http://www.movingtofreedom.org/2007/02/16/howto-remote-desktop-with-vnc-in-ubuntu-edgy-gnu-linux/

Or just do this:

System->Administration->Login Window
Remote->Style: same as local
Configure XDMCP: disable "Honor indirect requests"

Edit /etc/X11/gdm/gdm.conf for Feisty Fawn or /etc/gdm/gdm.conf for Gutsy Gibbon:
  • locate "[XDMCP]" and set "Enable=true"
  • locate and uncomment "RemoteGreeter=/usr/lib/gdm/gdmlogin"

sudo apt-get install vnc4server xinetd
sudo vncpasswd /root/.vncpasswd

Edit /etc/xinetd.d/Xvnc to add (.... signifies an indentation of 4 spaces):

service Xvnc
{
....type = UNLISTED
....disable = no
....socket_type = stream
....protocol = tcp
....wait = yes
....user = root
....server = /usr/bin/Xvnc
....server_args = -inetd :1 -query localhost -geometry 1024x768 -depth 16 -once -fp /usr/share/fonts/X11/misc -DisconnectClients=0 -NeverShared passwordFile=/root/.vncpasswd -extension XFIXES
....port = 5901
}

sudo /etc/init.d/xinetd stop
sudo /etc/init.d/xinetd start


Pay attention to the port number in Xvnc (usually 5901). On the Windows client side, add this local forwarded port to Putty's SSH tunnel (127.0.0.1:5901). And the router, if you have one, must forward this port.

To access the Linux server from your Windows PC, start Putty through port 22 and login. Then start your VNC viewer (I use TightVNC Viewer).

Project Golden Child

In June of 2007, I started a project to develop a website and web application. I was completely new to web development. I sought a cheap and quick approach. For cheap, I decided to go entirely Open Source. For quick, I chose to write my web application in Seaside and Squeak.

This blog presents some of my hard-won lessons. It has been very arduous sifting through tons of information on the Internet, much of which is either misleading, incomplete or inaccurate. It is very tough finding reliable knowledge. I've done the information vetting and gone through all the trial and error. I hope you will find the posts useful.

Here is a list of the software I used:
  • Ubuntu Linux "Feisty Fawn"
  • Apache 2
  • PostgreSQL 8.2
  • Seaside/Squeak
  • Darwin Streaming Server
  • Firestarter firewall
  • VNC and SSH for remote access
  • Proftpd for FTP access
FYI, my Seaside development system was a Core 2 Duo Windows Vista machine.