RTC Forums

Subscription => Support => Topic started by: garyk1968 on February 10, 2010, 04:39:48 PM



Title: Returning data in JSON format
Post by: garyk1968 on February 10, 2010, 04:39:48 PM
Hi

I am using RTC to develop a lightweight HTTP server which can process REST style requests, query some data and return that data in JSON format. Now if I query using a browser I can get it to work but I need to consume this data on an iphone.

I can get my iphone code to retrieve JSON data from any number of webservices out on the web but not from my RTC webserver so it must be doing something different hence no resultset.

I am using RtcHTTPServer and TRtcDataProvider doing the querying and conversion of the data into JSON myself and simply doing a Write(<jsonstring>) but there must be something else that needs setting perhaps?

Any help most appreciated

Gary


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 10, 2010, 05:12:11 PM
If your app is working with browsers, my guess would be that you aren't setting some HTTP header values which are required by phones. For example, the Response['Content-Type'] is one of the things most browsers will try to automatically detect, but I doubt you can expect the same from apps running on a mobile device.

There are also other HTTP header values which the app running on a mobile device might expect from a Server. If you use the File_Client demo to request any file from a WebServer running Apache, you will see that the Server is returning values for 'SERVER', 'LAST-MODIFIED', 'ACCEPT-RANGES', 'DATE' and 'ETAG' in its HTTP header, all of which can be set by using the TRtcDataServer(Sender).Response property in the RTC SDK before using WriteHeader or Write to send out the actual content.

Here is an example of HTTP header sent from an Apache Web Server:
DATE: Wed, 10 Feb 2010 16:07:38 GMT
SERVER: Apache/2.2.14 (Unix)
LAST-MODIFIED: Wed, 02 Dec 2009 10:07:07 GMT
ETAG: "573d40b-5810-479bc0981c8ca"
ACCEPT-RANGES: bytes
CONTENT-LENGTH: 22544
CONTENT-TYPE: text/html

You can also use the File_Client demo to send a request to a JSON Server which is working with your iPhone to see what the Server is sending out. The problem could also be in your formatting of the JSON content (sent out with the Write method).

I hope this helps.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 10, 2010, 05:41:55 PM
I've taken a short look at the JSON format (Java Script Object Notation) definition and from what I see, it is a rather complex format. And this means that some format elements you are using in your response might either be missing or badly implemented on the implementation running on your iPhone, while those used by other JSON Servers are implemented correctly on the iPhone app responsible for receiving the data.

In other words, if you can't get your JSON response to be translated correctly on the iPhone, it could be due to a number of reasons. Bad data formatting on your side, which might be tolerated by some more complex implementations and thus work from Web Browsers running on PCs (for example, JSON classes you are using to prepare the data before sending might be doing something wrong). Or, you are using some format elements which aren't implemented on the iPhone app, or ... it's simply a fact of missing HTTP headers which are expected by the iPhone app.

Anyway ... if you do find out what is wrong (either in your code or in the RTC SDK), please let us know.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 10, 2010, 06:57:38 PM
Hi Danijel,

Thanks for your response. Yes I thought it might be related to headers, I am now setting the status to 200 OK and the content type to application/json but it still doesnt work. There is a method NSString stringWithContentsOfURL which should return the JSON result and store it in a string, this is before parsing but the string is always nil, Ive been using jsonlint.com to validate my json anyway but I think there is something wrong in the response/request side of things.

I'll carry on and let you know what I find!

Regards

Gary


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 10, 2010, 07:46:38 PM
Hi Danijel,

Just as a update I cant get the file_client project to work with any known json services (which I do know work)

The error I always get is EWinSockException 'WSocketResolveHost: Cannot convert host address <website address>'

For example this URL will return JSON;

http://ws.geonames.org/citiesJSON?north=44.1&south=-9.9&east=-22.4&west=55.2&lang=de

but either putting the whole line in the server address field or splitting out the url into the server name and the parameters into the query parameters also returns an error, any ideas?

Many thanks


Gary


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 10, 2010, 08:56:15 PM
Works just fine here. You are probably splitting the parameters wrong.

Here is what you should enter in the File_Client demo to get the data from the URL you have provided ...

Server Port: 80
Server Address: ws.geonames.org
FileName: citiesJSON
Query parameters: north=44.1&south=-9.9&east=-22.4&west=55.2&lang=de

When I enter the above parameters in File_Client demo, I get this RESPONSE header ...

Status: 200 OK
DATE: Wed, 10 Feb 2010 19:52:49 GMT
SERVER: Apache/2.2.10 (Linux/SUSE)
CACHE-CONTROL: no-cache
ACCESS-CONTROL-ALLOW-ORIGIN: *
VARY: User-Agent
TRANSFER-ENCODING: chunked
CONTENT-TYPE: application/json;charset=UTF-8

And this content body ...

{"geonames":[{"fcodeName":"capital of a political entity","countrycode":"MX","fcl":"P","fclName":"city, village,...","name":"Mexico City","wikipedia":"de.wikipedia.org/wiki/Mexiko-Stadt","lng":-99.1386111,"fcode":"PPLC","geonameId":3530597,"lat":19.4341667,"population":11285654},{"fcodeName":"capital of a political entity","countrycode":"PH","fcl":"P","fclName":"city, village,...","name":"Manila","wikipedia":"","lng":120.9822222,"fcode":"PPLC","geonameId":1701668,"lat":14.6041667,"population":10444527},{"fcodeName":"capital of a political entity","countrycode":"BD","fcl":"P","fclName":"city, village,...","name":"Dhaka","wikipedia":"de.wikipedia.org/wiki/Dhaka","lng":90.4086111,"fcode":"PPLC","geonameId":1185241,"lat":23.7230556,"population":10356500},{"fcodeName":"capital of a political entity","countrycode":"KR","fcl":"P","fclName":"city, village,...","name":"Seoul","wikipedia":"de.wikipedia.org/wiki/Seoul","lng":126.9997222,"fcode":"PPLC","geonameId":1835848,"lat":37.5663889,"population":10349312},{"fcodeName":"capital of a political entity","countrycode":"ID","fcl":"P","fclName":"city, village,...","name":"Jakarta","wikipedia":"de.wikipedia.org/wiki/Jakarta","lng":106.8294444,"fcode":"PPLC","geonameId":1642911,"lat":-6.1744444,"population":8540121},{"fcodeName":"capital of a political entity","countrycode":"JP","fcl":"P","fclName":"city, village,...","name":"Tokyo","wikipedia":"de.wikipedia.org/wiki/Tokio","lng":139.691677093506,"fcode":"PPLC","geonameId":1850147,"lat":35.6895265930799,"population":8336599},{"fcodeName":"capital of a political entity","countrycode":"TW","fcl":"P","fclName":"city, village,...","name":"Taipei","wikipedia":"de.wikipedia.org/wiki/Taipeh","lng":121.525,"fcode":"PPLC","geonameId":1668341,"lat":25.0391667,"population":7871900},{"fcodeName":"capital of a political entity","countrycode":"CN","fcl":"P","fclName":"city, village,...","name":"Beijing","wikipedia":"de.wikipedia.org/wiki/Peking","lng":116.397228240967,"fcode":"PPLC","geonameId":1816670,"lat":39.9074977414405,"population":7480601},{"fcodeName":"capital of a political entity","countrycode":"CO","fcl":"P","fclName":"city, village,...","name":"Bogotá","wikipedia":"de.wikipedia.org/wiki/Bogot%C3%A1","lng":-74.0833333,"fcode":"PPLC","geonameId":3688689,"lat":4.6,"population":7102602},{"fcodeName":"capital of a political entity","countrycode":"HK","fcl":"P","fclName":"city, village,...","name":"Hong Kong","wikipedia":"","lng":114.15007352829,"fcode":"PPLC","geonameId":1819729,"lat":22.2840136009625,"population":7012738}]}


Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 10, 2010, 09:09:39 PM
Btw ... here is the code you would use in Delphi to prepare a TRtcHttpClient and TRtcDataRequest components if you wanted to send the above example request to the above example Server ...

with RtcHttpClient1 do
  begin
  ServerPort:='80';
  ServerAddr:='ws.geonames.org';
  AutoConnect:=True;
  end;
with RtcDataRequest1 do
  begin
  Client := RtcHttpClient1;
  Request.Host:='ws.geonames.org';
  Request.Method:='GET';
  Request.FileName:='/citiesJSON';
  Request.Query.Text:='north=44.1&south=-9.9&east=-22.4&west=55.2&lang=de';
  Post;
  end;

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 10, 2010, 10:11:17 PM
Hi Danijel,

Thanks for your assistance it is most appreciated, I still cant get a valid result from file_client, I will try it on another machine in the morning but I get a 404 error as in the screenshot below

(http://www.domorewithsage.com/downloads/screenshot.jpg)

Best regards


Gary


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 10, 2010, 10:54:21 PM
Looks to me like you are connecting to a different physical Server. The Server you are connecting to is reporting "Apache/2.2.14 (Unix)" in the SERVER header, while the one I am connecting to reports "Apache/2.2.10 (Linux/SUSE)" at the same place, so these can't be the same Servers.

If you are going through a Proxy, the problem could be that the Proxy you are using is incorrectly handling the request. Or, the problem could be with your ISPs DNS Server, in which case the IP address resolution is mismatched.

The domain name "ws.geonames.org" is translated to the IP address 188.40.33.19 by my DNS Server. This is what the new File_Client shows as IP address and is also what I get when I do a PING ws.geonames.org using CMD on Windows.

Try doing a PING ws.geonames.org in command prompt (cmd.exe on Windows XP and later) and see if you get a different IP than me. If you do, try using the IP address 188.40.33.19 in the Server Address field of the File_Client demo and make sure to correct the Host Name after you've modified the Server Address, because the Host Name has to be "ws.geonames.org".

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 10:22:32 AM
Since you also have a problem connecting to a JSON Server running on the Internet, I am starting to think that your problem is not even the JSON output of your Server, but that your problem is caused by your firewall or wrong entry in your Internet Service Providers DNS Server.

Are you sure that your iPhone is getting through to your Server, opens a connection on your Server and sends a request?

You can easily test if you Server is accessible from your iPhone by using a Web Browser (Safari?) from your iPhone, regardless of the JSON implementation. Simply drop another rtcDataProvider component on the Server, link it to the RtcHttpServer component and implement its OnCheckRequest and OnDataReceived events as shown in the Quick Start Server lesson 1 to get a simple Server output which can be tested with any WebServer, then run the Server and use your Servers IP address in the WebBrowser on your iPhone.

You can also test your JSON code by placing a breakpoint in the OnCheckRequest event of your Server if you can run the Server in Delphi debugger. Or, if the Server is running on a remote PC where you do not have Delphi installed, you can use the rtcLog unit, call StartLog from your Servers initialization section and use the Log function from the OnCheckRequest event of your rtcDataProvider component to write what requests you are getting in. You can also add logging to the event where you are writing the result out, to see what your Server is sending out.

Also, regardless of your firewall and router settings, you should be able to connect to your Server from your iPhone directly by using your Servers IP address inside LAN. For this to work, you need WLAN and have to set up your iPhone to use WLAN for internet access. This would allow you to focus on getting your Server-side code working with the code used on your iPhone first.

Once you get this working, it is simply a matter of correctly setting up your firewalls to make your Server accessible over the Internet. If a Server works inside LAN, it has to work over the Internet - provided you set up your firewall/router correctly.

You can always test your implementation locally by using a Web Browser. If a HTTP Server is working over "localhost", provided your Server is accessible over the Internet (firewalls set up correctly), your Server-side code will work over the Internet just like it does locally. And once you have made sure that your iPhone client is able to connect to your RTC Server and gets a response from the Server, it's only a matter of correctly using JSON on both sides (Server and Client). If you are manually generating JSON data, the problem could be your code preparing the JSON response. But it could also be the JSON client implementation you are using on the iPhone.

Anyway ... please keep me posted on your progress and let me know if you need more info or help.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 11, 2010, 02:21:22 PM
Hi Danijel,

Its not a firewall issue (I dont think) as I am echoing the requests/query params to a memo field on my server and I can see the requests coming in ok, its just not sending anything back. I'm going to play around with the flie client demo because I can use that to query my server and at least see the response headers and will let you know. Rather than use my own server I will also amend the demo one, I'm not sure if I need to hook up the data provider events to the server events and using the demo server should at least ensure everything is setup correctly

Best Regards

Gary


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 03:51:37 PM
1) Is your rtcDataProvider component linked to your RtcHttpServer component (through its "Server" property)?

2) Have you implemented the "OnCheckRequest" event on your rtcDataProvider component and are you calling "Accept" in that event to accept the request which you want to process?

3) Have you implemented the "OnDataReceived" event on your rtcDataProvider component to wait until the complete request has been received from the client (Request.Complete=TRUE) before you start processing the request and sending a response out?

4) What do you mean by "not sending anything back"? If you use the Write method from the Sender:TRtcConnection parameter in the OnDataReceived event, data will be sent out. Is it possible you are calling the Write() procedure from Delphi instead of using the Write() method from the Sender:TRtcConnection component?

Can you post the code of all the events you have implemented on your rtcDataProvider component, so I can see if there is anything wrong there? I am still making only wild guesses and will not be able to help you unless I know what you are doing.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 04:23:27 PM
I am officially confused. In your very first port, when you have started the topic, you wrote <quote> ... if I query using a browser I can get it to work ... </quote> and in your last post you wrote <quote> ... I am echoing the requests/query params to a memo field on my server and I can see the requests coming in ok, its just not sending anything back.</quote>

Can you please clarify:

A) Is your Server really working with a Web Browser? In other words, if you enter something like http://localhost/myURI?x=myparams in the Address field of the Web Browser (I do not know what you are doing, so the URL is only an example), does the Web Browser get a response from the Server which you can save to a file and the file you receive contains the JSON data?

B) When you send a request from your iPhone, does the request get accepted in your OnCheckRequest event and does your Server send a response out the same way it does when the request comes in from a Web Browser?

If you can use a memo to print out request parameters, you can also use it to print out the data you are about to send out with the Write() method. As suggested, you can also use the rtcLog unit with its StartLog and Log methods to add more logging to your code.

If your iPhone is sending the same request to your Server as you Web Browser, and your Server works with the Web Browser by sending out JSON data, then your Server will also send the same response to your iPhone (provided the Server gets the request from the iPhone). There is no code in the RTC SDK which would discriminate one browser while working with another one. Unless you explicitly write something in your own code to NOT accept a request coming from one client, or to send out a different response to a client (depending on the request headers or IP or whatever).

I've just used Safari on my iPhone to browse through web pages hosted by a RTC Server running on my PC. I see no reason why this should not work. If your response is correctly formatted (you've said that you have validated your JSON data with some online app) and your Server is sending the response out to a Web Browser, and you say that your Server is getting requests from your iPhone, but the iPhone app is not receiving a response, then the problem can only be somewhere in the JSON client on the iPhone.

Or am I missing something?

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 04:40:13 PM
Is it possible that the client app you are using on your iPhone is working ONLY with HTTP/1.0 and does NOT support the new HTTP/1.1 standard?

The current implementation in the rtcHttpServer is always sending a HTTP/1.1 response out, even if it received the request using HTTP/1.0. All the clients I have tested so far, even those working only with HTTP/1.0, have not had a problem with this. But if the implementation on your iPhone is very lazily written, it could be waiting for the HTTP/1.0 text in the response.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 11, 2010, 05:33:21 PM
I am officially confused. In your very first port, when you have started the topic, you wrote <quote> ... if I query using a browser I can get it to work ... </quote> and in your last post you wrote <quote> ... I am echoing the requests/query params to a memo field on my server and I can see the requests coming in ok, its just not sending anything back.</quote>

Sure and the last thing I want to do is confuse you!

Can you please clarify:

A) Is your Server really working with a Web Browser? In other words, if you enter something like http://localhost/myURI?x=myparams in the Address field of the Web Browser (I do not know what you are doing, so the URL is only an example), does the Web Browser get a response from the Server which you can save to a file and the file you receive contains the JSON data?

Exactly that, I can enter the URL into a browser and it did show the JSON in the browser. Now I have used Response.ContentType and set it to 'application/json' it now prompts me for a filename, I save the file and in that file is json.

B) When you send a request from your iPhone, does the request get accepted in your OnCheckRequest event and does your Server send a response out the same way it does when the request comes in from a Web Browser?

In my OnCheckRequest I am simply writing to a memo field the FilePath.Value[0] and the Query.Value['index'] so when the iphone sends the request the url is http://<server ip>/get/customers?index=A I see two lines in the memo field 'get' and 'A'

If you can use a memo to print out request parameters, you can also use it to print out the data you are about to send out with the Write() method. As suggested, you can also use the rtcLog unit with its StartLog and Log methods to add more logging to your code.

Well I have used your file_client demo and it returns the JSON as well as 200, OK and content-type:application/json from my server but I will add rtcLog as well.

If your iPhone is sending the same request to your Server as you Web Browser, and your Server works with the Web Browser by sending out JSON data, then your Server will also send the same response to your iPhone (provided the Server gets the request from the iPhone). There is no code in the RTC SDK which would discriminate one browser while working with another one. Unless you explicitly write something in your own code to NOT accept a request coming from one client, or to send out a different response to a client (depending on the request headers or IP or whatever).
The only thing I haven't tried is putting the url into safari on the mac I will do that. I run windows in virtual machine on the mac so have been using that to build/test the server from within the virtual machine to itself (i.e. localhost) as well as to another windows box but like I say I can see the requests coming in from the iphone anyway.

I've just used Safari on my iPhone to browse through web pages hosted by a RTC Server running on my PC. I see no reason why this should not work. If your response is correctly formatted (you've said that you have validated your JSON data with some online app) and your Server is sending the response out to a Web Browser, and you say that your Server is getting requests from your iPhone, but the iPhone app is not receiving a response, then the problem can only be somewhere in the JSON client on the iPhone.

Or am I missing something?

Well its not getting as far as the JSON parsing. In objective C strings are objects (NSString) and you simply create an instance of NSString with the initWithContentsOfURL method which creates your instance, goes off to the URL you specify and returns your result, be it HTML, XML or JSON it doesn't care. Once your NSString has the data then you do whatever parsing you need to do. And this is where it makes no sense, calling the webservice I showed you above it works but with RTC it doesn't. I'm not blaming RTC at all, its likely to be me but I'm darned if I can find out why! I will go through the steps you mentioned in you previous post and check all settings/event handlers from the Delphi side and as you say browse on the device, I'm actually running this on the iphone simulator not the device but it shouldn't make any difference.

Thanks for your patience and help!

Gary



Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 06:12:27 PM
Can you please check if your iPhone app is sending the request using HTTP/1.0? To do this from your RTC Server, simply check the state of the Request.Close property. If it is TRUE, then the request was received using HTTP/1.0 and the client might be expecting a response in HTTP/1.0 but will receive a HTTP/1.1 response.

I am already preparing an update for the RTC SDK which will extend the rtcHttpServer component to always respect the HTTP protocol used by the client and respond using the same protocol version (as said, the current implementation always sends a HTTP/1.1 response out). If the client is sending a HTTP/1.0 request, the update which I am currently preparing should fix the problem.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 07:01:42 PM
RTC SDK 3.50 is now ready for download, including a fix for working with HTTP/1.0 clients which expect a HTTP/1.0 response and can not process a response if it was sent using the HTTP/1.1 header.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 11, 2010, 08:31:14 PM
1) Is your rtcDataProvider component linked to your RtcHttpServer component (through its "Server" property)?

Yes it is linked.

2) Have you implemented the "OnCheckRequest" event on your rtcDataProvider component and are you calling "Accept" in that event to accept the request which you want to process?

Yes the code is;
  with Sender as TRtcDataServer do
  begin
    Memo1.Lines.Add(Request.FileName);
    Memo1.Lines.Add(Request.Query.Value['index']);

    if Request.FileName = '/' then
       Accept;

    if UpperCase(Request.FileName) = '/GET/CUSTOMERS' then
       Accept;
  end;


3) Have you implemented the "OnDataReceived" event on your rtcDataProvider component to wait until the complete request has been received from the client (Request.Complete=TRUE) before you start processing the request and sending a response out?

Yes code is;
var
  sResult: string;
begin
  with Sender as TRtcDataServer do
  begin
    if Request.Complete then
    begin
      if Request.Filename = '/' then
        Write('<H1>Test Server Running</H1>');
      if UpperCase(Request.FileName) = '/GET/CUSTOMERS' then
      begin
        sResult := dmSRV.GetCustomers(Request.Query.Value['index']);
        //Response.ContentType := 'application/json;charset=UTF-8'; -commented out to test on screen vs saving to file
        Write(sResult);
      end;
    end;
  end;
end;

4) What do you mean by "not sending anything back"? If you use the Write method from the Sender:TRtcConnection parameter in the OnDataReceived event, data will be sent out. Is it possible you are calling the Write() procedure from Delphi instead of using the Write() method from the Sender:TRtcConnection component?

I mean that nothing is coming back to the iphone application, I am debugging it and setting a breakpoint to test the return value which is always nil.

I have tried the url via the safari browser on the iphone simulator and that just throws up a 'safari cannot download this file' error. I have disabled the firewall on the windows box (testing it against server on the separate box rather than the windows running on vmware on the mac) running out of things try now :(

Best regards


Gary


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 09:07:49 PM
Even though I would recommend using a separate rtcDataProvider component for the '/' request to avoid having to check the Request.FileName value in the OnDataReceived event, your implementation is generally fine the way it is now. I'd just change the response for the '/' request to '<html>Server running</html>' instead of using '<h1></h1>' to make sure the Browser recognizes it as a HTML page.

Are you able to get a web page displayed in Safari if you just enter the IP address of your Server PC in Safari's Address field? Since you have the '/' request handler implemented, this should work regardless of your JSON implementation.

If you see that your Server is being accessed (requests pop up in Memo1), my guess is that your problem is either related to the HTTP/1.1 vs HTTP/1.0 issue, which I've mentioned above and have fixed in the latest RTC SDK 3.50 update, or ... it's a problem of the iPhone emulator not being able to connect to your PC.

To eliminate the HTTP/1.0 vs HTTP/1.1 issue (or confirm it?), please extend your OnCheckRequest event to print out the state of the Request.Close property, for example like this ...

with Sender as TRtcDataServer do
  begin
    Memo1.Lines.Add(Request.FileName);
    Memo1.Lines.Add(Request.Query.Value['index']);
    if Request.Close then
      Memo1.Lines.Add('!!! HTTP/1.0 !!!');

    if Request.FileName = '/' then
       Accept;

    if UpperCase(Request.FileName) = '/GET/CUSTOMERS' then
       Accept;
  end;

... And let me know if HTTP/1.0 will be added in the memo when a request comes from iPhone.

If the Server gets a request from the Client (you see it added to Memo1), then your problem has nothing to do with firewalls. A firewall would completely block communication and you would not get anything on the Server.

So ... before we continue, can you please let me know:

1) Is your Server receiving a request from your iPhone app and logging in in Memo1?

2) If your Server is receiving a request from your iPhone app, is the value of the Request.Close property TRUE or FALSE in the OnCheckRequest event when the request is received from iPhone?

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 09:31:18 PM
I have Xcode and the iPhone Simulator Version 3.0 (138) installed on my MacMini (running OSX 10.5.8), so I've tested if the iPhone Simulator has problems accessing the LAN, but that is not the case. Safari running in the iPhone Simulator was able to connect to a RTC Server running on my main PC (which is running Windows Vista) and display the web page provided by the RTC Server. But you've said you are running Windows inside a VM and VMs use virtual network cards, but if you can access your RTC Server from Safari on Mac OSX, you should be able to access it from Safari running inside the iPhone Simulator. If you can't access your RTC Server from Safari running on Mac OSX, the problem could be in the way network is configured in your VM.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 11, 2010, 09:35:50 PM
Are you able to get a web page displayed in Safari if you just enter the IP address of your Server PC in Safari's Address field? Since you have the '/' request handler implemented, this should work regardless of your JSON implementation.

... And let me know if HTTP/1.0 will be added in the memo when a request comes from iPhone.




1) Is your Server receiving a request from your iPhone app and logging in in Memo1?

response gets received and I see the value of 'index' in the memo field from the iphone request

2) If your Server is receiving a request from your iPhone app, is the value of the Request.Close property TRUE or FALSE in the OnCheckRequest event when the request is received from iPhone?

and is coming in as http1.1 request.close is false


best regards


Gary


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 11, 2010, 09:38:57 PM
I have Xcode and the iPhone Simulator Version 3.0 (138) installed on my MacMini (running OSX 10.5.8), so I've tested if the iPhone Simulator has problems accessing the LAN, but that is not the case. Safari running in the iPhone Simulator was able to connect to a RTC Server running on my main PC (which is running Windows Vista) and display the web page provided by the RTC Server. But you've said you are running Windows inside a VM and VMs use virtual network cards, but if you can access your RTC Server from Safari on Mac OSX, you should be able to access it from Safari running inside the iPhone Simulator. If you can't access your RTC Server from Safari running on Mac OSX, the problem could be in the way network is configured in your VM.

Best Regards,
Danijel Tkalcec

Yes your right the VM does strange things which is why I am using a separate windows box. It even gives the VM an IP not in the range of my router which is strange. Anyway its now a simple communication between the iphone, my app or safari and another windows machine on my network running the server.


Just as an update I am now grabbing an error message which should help, from cocoa I am getting an error  261 which is a 'Read error because the string encoding was not applicable.' as I have to specify a type of string encoding and I am using UTF8 which works fine for other sites but this maybe the problem.

cheers

Gary


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 09:59:11 PM
Ok, so the request is sent from your iPhone app and received by the RTC Server. What about the response sent from the RTC Server to the iPhone app? Are you using some JSON-specific library to send the request and receive a response, or are you using general-purpose sockets or HTTP transports?

Can you modify your iPhone app to send a HTTP request to the Server and show the response (what ever received) in a scrollable text box (in the iPhone app), or do anything which would get us a step further to see if the code you are using on the iPhone is receiving a response but is unable to parse JSON data, or if the problem is really that no response is received by the app running inside your iPhone Simulator?

Can you post the code you are using on the iPhone side, be it in Objective-C, C, C++ or Objective-Pascal, so I can also do my own debugging and see what is happening, or have you already solved the problem and its all working now?

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 11, 2010, 10:07:45 PM
Ah, just read your updated reply. That is a big progress from "not receiving anything" ;)

You need to know that the RTC SDK does not automatically encode nor decode any data being sent nor received. You read and write data using the AnsiString type, so you need to do proper encoding and decoding.

If you are working with Delphi 2009 and later, which have Unicode support, and the code you are using on the iPhone understands UTF-8, you can use the Utf8Encode function from the rtcInfo unit to convert your UnicodeString to a valid UTF-8 encoded AnsiString. If you set the ContentType correctly ('application/json;charset=UTF-8' looks fine to me) and use Utf8Encode on the string before sending, it might do the trick:

Response.ContentType:='application/json;charset=UTF-8';
Write(Utf8Encode(sResult));

Please let me know if that solves your problem.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 12, 2010, 08:42:11 AM
Hi Danijel,

Finally! What was it? a freakin pound sign '£'!!

So this is valid

{"Results":[{"account_ref":"AAB010","name":"A A Barnfield","balance":"0.00"}]}

this errors

{"Results":[{"account_ref":"AAB010","name":"A A Barnfield","balance":"£0.00"}]}

So I have it working!

Many thanks for all your help and patience it is most appreciated. If you do dip your toe in the ObjC/Cocoa waters and need help feel free to ask me any questions at gary@nospam.domorewithsage.com obviously taking the nospam bit out. I'm a long term delphi dev (since D1) and it took me a while to get my head around iphone development as it is such a different approach.

I need to buy a licence for your product anyway as this will form part of a commercial product.

Thanks again


Gary


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 12, 2010, 10:41:07 AM
Finally! :)

I'm glad you've found and solved the issue, but I am wondering why a pound sign put inside a string would be a problem. From your reply, it looks like the value for "balance" was sent as a string and not as a numeric, which means that using a pound sign should be allowed.

A short look at http://www.json.org/ does make it look like schema definition is not absolutely necessary for parsing JSON data, which would mean that parsing "balance":"L0.00" should work, but a pound sign would probably have to be encoded. I'm not sure what to make of * Any UNICODE character except " or \ or control character *, but shouldn't Utf8Encode() do the trick when charset in ContentType is set to UTF-8?

In other words ... could you problem also be solved without removing the pound sign from the original string by using the Utf8Encode function from the rtcInfo unit to get it encoded to UTF-8 (provided the string does not have " or \ or a control character -> what ever that is)?

...
uses rtcInfo;
...
Write(Utf8Encode(sResult));
...


If you would be sending the value for "balance" as a number (without enclosing it inside quotation marks), a pound sign would not be allowed, but since you are sending it as a string !?!? hmm ... food for thought.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 12, 2010, 11:21:17 AM
In other words ... a pound sign was making you trouble in this particular data sample, but if you do not encode your strings properly, you will get more problems once you try sending other Unicode strings (for example, using the name 'Müller' for the field 'name' would most likely start the alarm again).

But using Utf8Encode on the string you are sending out would not handle all the cases. You will need to write a function for converting \ and " as well as other 'control characters' to the accepted JSON string format (see http://www.json.org ).

Also, to avoid conversions from string to numerics on the Client, you might want to send numeric values without quotes ...
{"Results":[{"account_ref":"AAB010","name":"A A Barnfield","balance":0.00}]}

I may be wrong here, but if I've understood the JSON format correctly, it looks like "Results" is an array but you are sending all your parameters as a single object inside the array, which means that you should also be able to use { } instead of [{ }] to send your result parameters (provided you do not expect more than one result) ...
{"Results":{"account_ref":"AAB010","name":"A A Barnfield","balance":0.00}}

Since you do not seem to be using the JSON-RPC format and Results is currently holding a single object, enclosing all parameters inside a separate "Results" object might also be unnecessary, so your result would probably also be valid if sent like ...
{"account_ref":"AAB010","name":"A A Barnfield","balance":0.00}

Naturally, all of this depends on what the client is expecting on the other side. If the client is expecting a Result array which could have more than one element and each element can be an object, then the full format you are using is necessary.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: D.Tkalcec (RTC) on February 12, 2010, 11:49:39 AM
I've just found the RFC 4627, which (according to http://en.wikipedia.org/wiki/JSON ) completely describes the JSON format:
http://tools.ietf.org/html/rfc4627

Interesting to note here are sections 2.5 and 3, which describe the String format and String Encoding.

Best Regards,
Danijel Tkalcec


Title: Re: Returning data in JSON format
Post by: garyk1968 on February 12, 2010, 12:59:50 PM
In other words ... a pound sign was making you trouble in this particular data sample, but if you do not encode your strings properly, you will get more problems once you try sending other Unicode strings (for example, using the name 'Müller' for the field 'name' would most likely start the alarm again).

Yes as soon as I had the encoding error I thought to myself I bet its to do with certain characters, it woke me at 6am and I had to go and see if that was the case!

But using Utf8Encode on the string you are sending out would not handle all the cases. You will need to write a function for converting \ and " as well as other 'control characters' to the accepted JSON string format (see http://www.json.org ).

I didnt get around to using the UTF8encode but I will put the pound sign back in and try it and see what happens!

Also, to avoid conversions from string to numerics on the Client, you might want to send numeric values without quotes ...
{"Results":[{"account_ref":"AAB010","name":"A A Barnfield","balance":0.00}]}

I may be wrong here, but if I've understood the JSON format correctly, it looks like "Results" is an array but you are sending all your parameters as a single object inside the array, which means that you should also be able to use { } instead of [{ }] to send your result parameters (provided you do not expect more than one result) ...
{"Results":{"account_ref":"AAB010","name":"A A Barnfield","balance":0.00}}

Since you do not seem to be using the JSON-RPC format and Results is currently holding a single object, enclosing all parameters inside a separate "Results" object might also be unnecessary, so your result would probably also be valid if sent like ...
{"account_ref":"AAB010","name":"A A Barnfield","balance":0.00}

Yes I wasnt sure whether to include the numeric in quotes or not, I need to re-read the JSON specification.

As to regards {} instead of [{}] the result set is returning multiple rows I just edited to contain one record but you are correct for a single entry you *could* do that although I am iterating through an array of 1..n elements based on what is inside the [].

As for your propsoal of supporting JSON that would be great although its quite easy to construct it using a normal TDataSet descendant just iterating through the result set and building the string.

Best Regards


Gary