Presence and UDP Holepunching problems..

Hello!

It has been a while, but now we have to write again, so dont miss out anythihng.
Today we have implemented the presence (XEP-115) that are needed to show that a client have voice capabilities. It was not that easy that it seems. We needed to have two lists to order and to keep into place. This is because you can have more than one client connected with different resources and different presence states.
Until now we have implemented this and have not taken priority into any regards. Maybe we have to look into this later, because if you have two clients: one with "available" and one with "away". As it is now it "hopefully will be routed to the available one.

We also have to loook into presence when we are talking voice to each other. Maybe we should signal out a busy signal (it could be nice to have). XEP-167 also descripes the SDP (session description protocol). We SHOULD also implement this, so we can refuse a voice session and present it nicely to the user.

BTW: we set the service discovery (XEP-030) to make our client voice-aware (xep-166, xep-167 name space). It does not seem like that gtalk are using these. It might (and probably does) use some of their own name space. But for now we cant find out what they are using. And in the XMPP standards they are writing that presence is more useful than the service discovery. So for now we have implemented xep-030 for xmpp and is using the precence oriented way of showing our voice service.


UDP Hole punching is problem right now. We have made STUN discovery of our own public IP and port and it works out fine after a couple of small hacks in their core stun.c.
But now we need to use these public endpoints to send out UDP fake packages to the different candidates and to punch hole with several UDP packages. After this we will return the actual socket file descriptor from the session and give it to the gstreamer voice_server.c. And right now it is the latter that does not work.
More investigation for parameters for GST is neeeded.

More STUN..

For now, we are able to get the public IP and the public port. The public port was not easily harvested through the pidgin API, so we had to hack the stun.c and stun.h for altering the structure and the method for retrieving these.

At the same time we also found out that we are not able to punch hole through our NATs and therefore the API is probably not have this functionality. We have to implement this ourselves. We found some good reading about UDP hole punching which is a technique that allows traversing NATs. Locally our code is working fine. Though still some comparing neeeds to be done, so we are able to see is the peers are locally available or not.

We also did some near implementation of candidates, so our voice code takes candidates as parameteres instead of alot of diffrent parameters such as ipaddr, port etc. We also made a implementation so that we can contain two different list in the voicenode structure: local_candidates and peer_candidates. This need also to be done for the payload!

Also we did a milestone document.

NAT and STUN..

Threading is almost like done now, and the synchronization of the GUI, the voice threads and the protocol backend seems to work out fine now.
Actually we have been looking at the NAT/STUN area for a while now. Most of it in theory we have been looking at NAT traversal techniques etc.

• STUN (for getting the public IP and port)
  
• UPnP (xml webservice from and to the router)
• TURN
• ICE (which is a specific combination of some of the techniques above)

But for the code matters, we have been trying to use the already implemented STUN API in pidgin (stun.c, stun.h, network.c, network.h). We found that some of the stuff have the possibility to get the public IP and the ability to listen and maybe punch hole from inside out of the router. Right now we have patched it go set and get the port for the STUN server when connecting to it. At the original implementation it was hardcoded to the "normal" STUN server port (UDP port 3478). Googles STUN servers listening on UDP port 19xxx.

But when we find out if we should expand and patch the pidgin STUN API or not (I think we shall), then we should be able to get voice through our pidgin, even if they are behind NAT!. This means that we will get the correct IP addresses to send via. XMPP, to have punched a hole in the NAT and finally also have advertized the correct public IP and port. We also made the XMPP/GTalk implementation of STUN discovery, which is documented here:

• http://code.google.com/apis/talk/jep_extensions/jingleinfo.html
• http://www.xmpp.org/extensions/xep-0215.html

This worked fine, we got all the google STUN servers when requesting. We also here found out, that they used another port, than the usual STUN servers (look earlier statement).

Damn I am looking forward to these coming weeks!...

But when this "small" problem is fixed we still have a long way to go, we still need the functionality of:

• Be able to see a busy signal of the line (and a busy dial tone, ha ha).
• To hold a line through the GUI (if you dont have time just now)
• Implement OTR (of the record, and store a voice mail in google mail)
• implement ID in the xmpp/gtalk. We need this in order to have complete control, if more than one is calling etc.
• 
  
• fix small bugs, when hanging up, making more than one call
  
• make detailed description (XEP-xxx) of new voice conferingcing. This is not a XEP yet. We have to make it.
  
• make video implementation! ;-)
• A lot more, I cant remember..

Over and out....
/zool

Threading

The last to days have been spent on studying and implementing pthreads to our code. We need this to open a voice connection - while the rest of Pidgin keeps running.

Since it can be attempted to open several calls, or people can attempt to call you while in conversation, we must control the threading.
For now we have decided to spawn a maximum of 1 voice_client thread and 1 voice_server thread. That means we must make sure that we do not spawn to many threads, and that we notify where appropiate when a new call can not be done... (But what about calls on hold and conferencing... save that for later ;o) )

It seems to be working and synchronized using signals with the core of the plugin.

Sounds better

Besides all the official work (see previous post) - we have worked a lot on the code lately.

- We have incorporated our GStreamer code into our plugin. And we have got some sound voice through on our local network using hardcoded ip addresses. On the way we have had some include file problems, which seems to be over for now.
We have some bugs in GStreamer which we have to attend to and we still need to look at those IP-address problems. The GStreamer code is currently being synchronized with the XMPP code using signals.
- Lately we have researched NAT a lot, to address the IP problems.
- We have split up the GUI from the rest of the code even more - using signals. More is still to be done here...

Delivery Date

Now it has been settled. The 14th of March we have to be done with this master thesis.
Actually we aim to finish before the Winter holiday mid February.

(Before) Next week we will set up some milestones for the project.

Some Problems solved..

Hey Again,

The problems we had with gstreamer and using its elements are now solved. We had problems with the voice comming through the RTP protocol. That was actually because that the RTBbin that we used was been obsolete and deprecated. Now we use udpsrc etc. (see the source code), which acutally make the voice go through over RTP!. *GREAT!*
Now we are only missing to incoorporate it into our plugin code. This is not done yet because of some heavy research regarding NAT and getting the public IP address.

As I (zool) spend last week of researching how to bypass the NAT, I have found some mechanisms which I only will mention briefly here. I have also writte a section in the report about the same area:

*STUN
*TURN
*UPnP
*ICE
....

But better yet I found that google (gtalk) and xmpp finally have something in common now. This is pretty new I think:

http://www.xmpp.org/extensions/xep-0215.html
http://code.google.com/apis/talk/jep_extensions/jingleinfo.html

These links show how STUN servers are found via. XMPP. After we have found the server list, we can connect (Via. a STUN client) with one of them. The one thing I dont get is that I have seen many places that states that gtalk is using ICE. I have to investigate further than this. **13/8-2007* The day after (today), I already found out of the context of ICE. ICE is the technique for using STUN for getting the public IP addr. and using TURN for data traversal through nat.

I talked (via. IRC, #pidgin) with a lot of the developers for the pidgin platform, which have connections to the xmpp organization (Sean Egan, utopia etc), and found out that no standard or draft papers are available for conferencing of voice or video. There for now I am involved in making findings of already available software in this area, and find out what kind of demands that can be put. After this I will write a XEP-### for the draft papers.!!! Jubiii, we can be official xmpp developers.. .*GREAT*!...

Well thats all for now!

10-4
/zool (Steffen)

Summer problems...

Welcome back to us both. After a summerholliday we both looked forward to get into action with the project again. Unfortunately we came back to problems that we are still into.
It was time to look at the GStreamer library again and some ip configuration using uPnP STUN ICE or something.

We found that the network API for for Pidgin seems bugged.

We also found that the once working GStreamer RTPBIN now has become bugged after some changes in the GStreamer libraries.
So the last couple of weeks have been spent on reading mail-archieves, looking at source code, writing forums and turning gray haired!

Basically we are very annoyed...