cURL : how to read mail ?

Discussions related to Visual Prolog
User avatar
Tonton Luc
VIP Member
Posts: 501
Joined: 16 Oct 2001 23:01

cURL : how to read mail ?

Unread post by Tonton Luc » 2 Jan 2015 13:41

Hi,

Here is my code :

Code: Select all

  startCase(Action) :-           Action = 2,             try                 ResultStream = outputStream_String::new(),                 Connection = vPcURL::new(ResultStream, caseCompletion, Action),                 _ = Connection:curl_easy_setopt(cURLOPT_USERPWD, s("my_email@my_domain.com:my_password")),                 _ = Connection:curl_easy_setopt(cURLOPT_URL, s("pop3://my_pop3_domain.com")),                 _ = Connection:curl_easy_perform()             catch _ do                 taskWindow::ecri_log(string::format("startCase % pas ok.", Action)),                 succeed()             end try.  
...but how to read the mail ? This following code just return a number (or a pointer ?) :

Code: Select all

    caseCompletion(cURLE_OK, 2, Connection) :-         ResultStreamS = uncheckedConvert(outputstream_string, Connection:getOutputStream()),         Received = ResultStreamS:getString(),         ResultStreamS:close(),         Received8 = uncheckedConvert(string8, Received),         Reponse = string8::mapToString(Received8),         Connection:curl_easy_cleanup(),         setCaseIdle(2),         stdio::write(Reponse),stdio::nl,         !.

User avatar
Jan de Lint
VIP Member
Posts: 107
Joined: 6 Mar 2000 0:01

Unread post by Jan de Lint » 14 Jan 2015 13:59

Hi Tonton,
just two remarks.
- nice touch adding the try/catch !
- it is not advisable to do so many unchecked converts since VIPcURL latest version supports binary strings. Perhaps using those helps.
Regards.
]an

User avatar
Tonton Luc
VIP Member
Posts: 501
Joined: 16 Oct 2001 23:01

Unread post by Tonton Luc » 14 Jan 2015 15:42

Hi Jan,

I don't understand what you mean. Can you explain-me more ?

I've changing my code :

Code: Select all

    caseCompletion(cURLE_OK, 2, Connection) :-         ResultStreamS = uncheckedConvert(outputstream_string, Connection:getOutputStream()),         Received = ResultStreamS:getString(),         ResultStreamS:close(),         Received8 = uncheckedConvert(string8, Received),         Reponse = string8::mapToString(Received8),         Connection:curl_easy_cleanup(),         setCaseIdle(2),         file5x::file_str("D:\\1travail\\Prolog_7_3\\Test_lib_cURL_7_3\\Exe\\result.txt",Reponse),         stdio::write("Ok"),stdio::nl,         !.
In the txt file, I can read a header of the mail but how to read the body of the mail ?

User avatar
Jan de Lint
VIP Member
Posts: 107
Joined: 6 Mar 2000 0:01

Unread post by Jan de Lint » 18 Jan 2015 20:40

Could be the POP3 server output will contain more than one string. Or perhaps you need to get the body in a second step.
From VPcURL.i:

Code: Select all

    getOutput : () -> string.  % the budget version (ANSI presumed)     getOutput : (codePage) -> string_list.     getOutputBin : () -> binary.
So try the other getOutputs.
If you want to know the layout of the POP3 server output it may be advisable to google around a bit. It's too long ago for me. VPcUrl proper does not modify the POP3 output.
Also, if you need to write a string to a file, use for instance

Code: Select all

       file::writeString("Filename", String ,false()), !.
Success,
]an

User avatar
Thomas Linder Puls
VIP Member
Posts: 1622
Joined: 28 Feb 2000 0:01

Unread post by Thomas Linder Puls » 18 Jan 2015 20:55

(Not really knowing anything about cURL) I would be very surprised if this line of code makes any sense:

Code: Select all

        ResultStreamS = uncheckedConvert(outputstream_string, Connection:getOutputStream()),
Something is not an outputStream_string "just like that".
Regards Thomas Linder Puls
PDC

User avatar
Jan de Lint
VIP Member
Posts: 107
Joined: 6 Mar 2000 0:01

Unread post by Jan de Lint » 19 Jan 2015 11:09

Yes indeed. The output of VPcURL is 'raw'. One has to apply some knowledge about the nature of the output.
Curl nor VPcURL has an automatic detection mechanism for for instance the code page or whether the output is binary or not.
'Ordinary' http pages consist of one single string - that being more or less the easy case. Other types of output must be hand-formatted to get the required result. Code page info can be contained in the headers, the user also has access to that info.
Regarding mail. Even professional mail clients used to make sometimes the wrong assumption regarding the intended code page.
]an

User avatar
Tonton Luc
VIP Member
Posts: 501
Joined: 16 Oct 2001 23:01

Unread post by Tonton Luc » 27 Jan 2015 8:43

Hi,

Here is the beginning of the respons when I read only the first mail with cURL :
Received: from smtp-1.planint.com ([83.165.37.24])

by mail.rd-cassis.net (rd-cassis.net) with ESMTP id 201501561657565548

for <xxx.xxxxx@jdklog.com>; Mon, 26 Jan 2015 16:57:56 +0100

Received: from mbprod2008web.neo.planint.com ([10.10.0.104] RDNS failed) by smtp-1.planint.com with Microsoft SMTPSVC(6.0.3791.4675);

Mon, 26 Jan 2015 16:57:57 +0100

Received: from MBPROD2008WEB ([127.0.0.1]) by mbprod2008web.neo.planint.com with Microsoft SMTPSVC(7.5.7601.17514);

Mon, 26 Jan 2015 16:57:57 +0100

MIME-Version: 1.0

From: "Mercedes-Benz" <contact@mercedes-benz.fr>

To: "Luc SABINE" <xxx.xxxxx@jdklog.com>

Date: 26 Jan 2015 16:57:57 +0100

Subject: M. SABINE, votre documentation Mercedes-Benz

Content-Type: text/html; charset=utf-8

Content-Transfer-Encoding: base64

Return-Path: contact@mercedes-benz.fr

Message-ID: <MBPROD2009WEBdk4RA5000021a5@mbprod2008web.neo.planint.com>

X-OriginalArrivalTime: 26 Jan 2015 15:57:57.0561 (UTC) FILETIME=[DA590A90:01D03980]

X-Spam-Status: No, hits=-1,18 required=6,40 tests=MIME_HTML_ONLY=0,72,HTML_MESSAGE=0,00,BAYES_00=-1,90

version=3.3.2

X-Spam-Level:

X-Spam-Checker-Version: SpamAssassin 3.3.2 (1.1) on mail.rd-cassis.net

X-Spam-IndexStatus: 0



PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMDEgVHJhbnNp

dGlvbmFsLy9FTiIgImh0dHA6Ly93d3cudzMub3JnL1RSL2h0bWw0L2xvb3NlLmR0ZCI+

DQo8aHRtbD4NCg0KPCEtLQ0KREFHIE1lcmNlZGVzLUJlbnogQkMvTUkgQ1JNIEFrcXVp

c2l0aW9uICYgTG95YWx0eSBlTmV3c2xldHRlciBUZW1wbGF0ZQ0KVmVyc2lvbjogMS4x

IC0gcGxlYXNlIHNl.....
I need to recover the body of the mail => I supose it's the part started by "PCFET0NUWVB...".
Any way to convert this part ?

User avatar
Thomas Linder Puls
VIP Member
Posts: 1622
Joined: 28 Feb 2000 0:01

Unread post by Thomas Linder Puls » 27 Jan 2015 10:25

You seem to have two line shifts each time you should have one (perhaps something is considering both "Carriage Return" and "Line Feed" as line shifts). Anyway the SMTP (and a lot of other internet stuff) have a number of headers separated by <CR><LF>. The headers are terminated by an empty header (or depending on how you want to see it two consecutive <CR><LF> sets).

Two interesting headers are:
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64
I believe the first one is quite obvious, the second means that the contents is encoded using base64.

Assuming that you have extracted the part after <CR><LF><CR><LF> you can get the HTML text like this:

Code: Select all

class predicates     decode_base64_utf8 : (string Base64Uft8) -> string Decoded. clauses     decode_base64_utf8(Base64Uft8) = Decoded :-         Utf8Binary = cryptography::base64_decode(Base64Uft8),         Utf8 = uncheckedConvert(string8, Utf8Binary),         Decoded = string8::fromUtf8(Utf8).
(The cryptography package is only in the Commercial Edition).
Regards Thomas Linder Puls
PDC

User avatar
Tonton Luc
VIP Member
Posts: 501
Joined: 16 Oct 2001 23:01

Unread post by Tonton Luc » 27 Jan 2015 10:50

Hi,

Sorry but with VP 7.3 cryptography class have only base64_encode (not base64_decode).
Any other way ?

User avatar
Tonton Luc
VIP Member
Posts: 501
Joined: 16 Oct 2001 23:01

Unread post by Tonton Luc » 28 Jan 2015 13:42

:idea: Using this following VP 7.3 code, all works fine :

Code: Select all

implement decrypt     open core   constants     className = "TaskWindow/decrypt".     classVersion = "$JustDate: $$Revision: $".     base64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/".   class facts numDataBytes:integer:=erroneous. nGroup:integer:=erroneous. thisChar:string:=erroneous. thisData:integer:=erroneous. sOut:string:=erroneous.   clauses     classInfo(className, classVersion).         base64decode(S) = Result:-         sOut := "",         S1 = string::replaceAll(S,"\r\n",""),         S2 = string::replaceAll(S1,"\t",""),         S3 = string::replaceAll(S2," ",""),         Len = string::length(S3),         Len mod 4 = 0,         foreach GroupBegin = std::fromToInStep(1,Len,4) do             numDataBytes := 3,             nGroup := 0,             foreach CharCounter = std::fromTo(0,3) do                 thisChar := string::subString(S3,GroupBegin - 1 + CharCounter, 1),                 if thisChar = "=" then                     numDataBytes := numDataBytes - 1,                     thisData := 0                 else                     thisData := cherch_pos()                 end if,                 if not(thisData = -1) then                     nGroup := (64 * nGroup) + thisData                 end if             end foreach,             NGroup = string::format("%x",nGroup),             ajoute_zero(NGroup,NGroup_ok),             decoup_3_car(NGroup_ok,"",Suite),             sOut := string::concat(sOut,Suite)         end foreach,         Result = sOut,                 !.     base64decode(_) = "".   class predicates cherch_pos:()->integer. ajoute_zero:(string,string) procedure(i,o). decoup_3_car:(string,string,string) procedure(i,i,o). clauses cherch_pos() = N:-     N = string::search(base64,thisChar),     !. cherch_pos() = -1:-     stdio::write(">>",thisChar,"<<"),stdio::nl.     ajoute_zero(Se,Ss):-     Len = string::length(Se),     if Len = 6 then         Ss = Se     else         Ss = string::concat(string::create(6 - Len,"0"),Se)     end if.     decoup_3_car("",S,S):-!. decoup_3_car(Source,Se,S):-     string::front(Source,2,Deb,Ss),     conversion5x::char_int(Lettre,toTerm(string::concat("0x",Deb))),!,     decoup_3_car(Ss,string::format("%s%",Se,Lettre),S).   end implement decrypt

Code: Select all

predicates     base64decode:(string) -> string procedure(i).

User avatar
Thomas Linder Puls
VIP Member
Posts: 1622
Joined: 28 Feb 2000 0:01

Unread post by Thomas Linder Puls » 28 Jan 2015 20:17

:-)
Regards Thomas Linder Puls
PDC

User avatar
Jan de Lint
VIP Member
Posts: 107
Joined: 6 Mar 2000 0:01

Unread post by Jan de Lint » 28 Jan 2015 21:21

Great! You did it!
Long time ago, I made an automatic IMAP mail client along the same line - using PHP libraries. I also remember that required a lot of low level coding. Unavoidable, I'm afraid.
But once it works, it's cool 8-)
]an

Post Reply