A few problems I have run into

Submitted by Anonymous on Thu, 10/06/2011 - 11:37

Hello there,

I am currently implementing a few P2P streaming/filesharing protocols in Splay (maybe Philipp Eittenberger has told you guys) and have encountered a few problems:

1.) I have successfully installed my own Splay System in Ubuntu 10.4 LTS. Everything seems to be working fine, except that I can only register one splayd at the controller. That is a hindering my local testing a bit. I would like to set up 2-10 nodes. I have found the hint to set the "production" flag to false, but it doesn't seem to work for me. The deamon log reads all right "Warning, production mode disabled..." but I can still only register one Node per IP. Maybe I am missing something?

2.) Since I want to evaluate and test a few different file dissemination protocols, it is mandatory to have some kind of file to distribute. In another thread it was mentioned, to download the file from an external host via HTTP GET, because the virtual filesystem could not be prepopulated yet. I plan on using this method, however, the posted code and the one in the manual is not working for me - it seems that restricted_io cannot be acquired.
Is there another way to achieve this or some kind of workaround?

3.) In a P2P protocol, chunks/parts of the files are sent between the peers. I am not very familiar with Lua, but I am searching for the "best" method of sending/receiving these chunks. It seems that RPC's coupled with an identity function would be a THE least complex way of dealing with this, however RPC's do not resemble the communicational model of such protocols very much. So I guess I will have to rely on sockets. I have seen that you have built upon the standard Lua socket modules, maybe you can give me a short advice on which socket and encoding to use and how to set it up / implement it with the "node" array (IP, port).

4.) Churn management. I understand that there is a churn management already implemented, but the only example I could find, was the small piece of code in the NSDI paper. Is there any more documentation about this topic? Furthermore I am not sure how to get this code working - I guess one needs to save it to a .lua file and submit it in the "trace" menu, when creating a new job with the web-interface?

Thank you very much in advance. So far I am really impressed with how easy using splay is. The functions are very high-level and concise and everything I tried so far has worked almost instantly!

Best wishes,
Daniel Thomä

‹ Problem with splayd 285 on SplayWeb Deadlink for Splayd ›

hi, daniel. a little addition

Submitted by valerioj on Thu, 10/06/2011 - 13:36.

hi, daniel. a little addition to valerio's comments:

about q1. do you see an error message when trying to register a second node with the same IP address? what does it say?

about q4. there is an example of churn trace file inside the package for the Splay Command Line Interface, which you can get from the Download page.

josé

Dear Daniel, thanks for your

Submitted by vschiavoni on Thu, 10/06/2011 - 13:29.

Dear Daniel,
thanks for your interest in Splay.
I'll reply point by point:

1) Have you followed the tutorial that we provide for your specific scenario? It is available here:
http://www.splay-project.org/tutorials/cluster

and we use it for our own cluster deployments. Let me know in case some of the steps do not work for you!

2) That is weird. Once you get the cluster deployed and properly setup, that code should work. If not, do you have a stack trace that I can investigate a bit more, including the code that you tried?
Also, which version of the controller/splayd are you using?

3) RPC is the default mechanism and the most straightforward to use. It's been used to implement bit-torrent like protocols, and it seems you can use it as well. We had a plan to provide a MessagePassing-oriented library on top of our own restricted sockets, but this remains a TODO for now (contributions are welcome btw;-) )

4) We have internal documentations and a fairly sophisticated tool to generate traces. We haven't completely finalized the best way to expose these tools to a more wider audience. We'll keep you posted about this final point as soon as possible. Nevertheless, i can say that a churn trace is a simple .txt file where for each line you provide its UP and DOWN moments (more up and down moments per node are possible). Once this trace file is ready (which can be generated with other tools, extrapolated from real-world traces for example), you submit it together with the .lua file implementing the protocol you want to test under churn.

Thanks for helping us building a better tool !

Valerio

Dear Valerio & José,thank

Submitted by Anonymous on Fri, 10/07/2011 - 16:28.

Dear Valerio & José,

thank you very much for your quick and helpful replies! I am using the newest versions of splayd, controller and web-interface on a single host (virtual VMWare Ubuntu 10.4 LTS on a Win7 x86 Professional System). I will try the script posted in the cluster tutorial and let you know, if I encounter any further problems.

Concerning the restricted_io: I have tested it on my own single-splayd system, as well as on the PlanetLab deployment. Among others, I tried the exact same code as posted here:
Link to post

It should be among the last jobs submitted via my account.

Regards,
Daniel

Dear Daniel, i've checked

Submitted by vschiavoni on Mon, 10/10/2011 - 10:09.

Dear Daniel,
i've checked again the code snippet.
If you run it on the cluster or on SplayWeb, simply comment the line:

-- local io = require("splay.restricted_io")

and everything will work.
The lib io is available by default as soon as splay.base is required.

best regards,
Valerio