接口中转stream传输 request/response

摘要:
第一个是cURLresource;第二个是通过选项CURLOPT_INFILE传给cURL的streamresource;第三个参数是最大可以读取的数据的数量。回调函数必须返回一个字符串,长度小于或等于请求的数据量。一般从传入的streamresource读取。返回空字符串作为EOF信号。phpfunctionreceiveResponse{var_dump;flush();ob_flush();//需要准确返回读取的字节数returnstrlen;}$ch=curl_init('http://jksong.cm/api.php?
php实现
CURLOPT_WRITEFUNCTION: for response ,把接口返回的结果拿回来,会进行多次回调,直到接口中的内容全部读完
CURLOPT_READFUNCTION :for request ,把要请求接口的参数数据写出去
CURLOPT_READFUNCTION    
    回调函数名。该函数应接受三个参数。第一个是 cURL resource;
    第二个是通过选项 CURLOPT_INFILE 传给 cURL 的 stream resource;
    第三个参数是最大可以读取的数据的数量。回 调函数必须返回一个字符串,长度小于或等于请求的数据量(第三个参数)。一般从传入的 stream resource读取。返回空字符串作为 EOF(文件结束) 信号。
CURLOPT_WRITEFUNCTION    
    回调函数名。该函数应接受两个参数。
    第一个是 cURL resource;
    第二个是要写入的数据字符串。数 据必须在函数中被保存。 
    函数必须准确返回写入数据的字节数,否则传输会被一个错误所中断。
A bit more documentation (without minimum version numbers):

-CURLOPT_WRITEFUNCTION
-CURLOPT_HEADERFUNCTION
Pass a
function which will be called to write data or headers respectively. The callback function prototype: long write_callback (resource ch, stringdata) The ch argument is CURL session handle. The data argument is data received. Note that its size is variable. When writing data, as much data as possible will be returned in all invokes. When writing headers, exactly one complete header line is returned for better parsing.The function must return number of bytes actually taken care of. If that amount differs from the amount passed to this function, an error will occur. -CURLOPT_READFUNCTION Pass a function which will be called to read data. The callback function prototype: string read_callback (resource ch, resource fd,long length) The ch argument is CURL session handle. The fd argument is file descriptor passed to CURL by CURLOPT_INFILE option. The length argument is maximum length which can be returned.The function must return string containing the data which were read. If length of the data is more than maximum length, it will be truncated to maximum length. Returning anything else than a string means an EOF. [Note: there is more callbacks implemented in current cURL library but they aren't unfortunately implemented in php curl interface yet.]

文件下载

client.php

<?php

function receiveResponse($curlHandle, $xmldata)
{
    var_dump($xmldata);
    flush();
    ob_flush();
    //需要准确返回读取的字节数
    return strlen($xmldata);
}

$ch = curl_init('http://jksong.cm/api.php?id=1&name=jksong');
curl_setopt($ch, CURLOPT_WRITEFUNCTION, "receiveResponse");
curl_exec($ch);
curl_close($ch);

api.php

<?php

for ($i=0;$i<10;$i++){

    var_dump(str_repeat("a", 20));
//var_dump(time());

    sleep(1);

    ob_flush();
    flush();
}

JAVA httpclient 实现

CloseableHttpClient client =HttpClients.createDefault();
        HttpGet httpGet = new HttpGet("http://jksong.cm/api.php");
        try (CloseableHttpResponse response1 =client.execute(httpGet)) {
            final HttpEntity entity =response1.getEntity();
            if (entity != null) {
                try (InputStream inputStream =entity.getContent()) {
                    //do something useful with the stream
                    //System.out.println(IOUtils.toString(inputStream));
BufferedInputStream bufferedInputStream = newBufferedInputStream(inputStream);
                    
                    int b = 0;
                    while ((b = inputStream.read()) != -1) {
                        System.out.println("===>"+(char)b);
                    }

                }
            }
        }
node

http

var http = require('http');
var body = '';

http.get("http://jksong.cm/api.php", function(res) {
    res.on('data', function(chunk) {
        body +=chunk;
        console.log(chunk);
    });
    res.on('end', function() {
        //all data has been downloaded
});
});

axios

var fs = require('fs');
var http = require('axios');
//获取远端图片
http({
    method: 'get',
    url: 'http://jksong.cm/api.php',
    responseType: 'stream'})
    .then(function(response) {
        //console.log(response.data.read);
        response.data.pipe(fs.createWriteStream('ada_lovelace.txt'))
    });

  文件上传

PHP

php中form-data不支持 php://input 方式读取,可以采用 binary或urlencode方式 直接上传文件

验证:

api.php

//可以实时读取最新上传的数据
$handle
= fopen("php://input", "rb"); $contents = ''; while (!feof($handle)) { $body = fread($handle, 8192); var_dump($body); file_put_contents("/tmp/a.txt", $body,FILE_APPEND); } fclose($handle);

curl请求

#注意发送速率,由于服务端缓冲区的存在,太小时不容易测出实时接收的效果
curl --limit-rate 100k -L -X GET 'http://jksong.cm/api.php' -H 'Content-Type: application/x-rar-compressed' --data-binary '@/tmp/file.data'
curl --limit-rate 100k -L -X GET 'http://jksong.cm/api.php' -H 'Content-Type: application/x-rar-compressed' --data-urlencode 'song@/tmp/file.data'

JAVA

request.getInputStream() 中的流可以实时获取,这样就可以边上传,边进行转发

如果获取 inputstream 原因参看 接口中转stream传输

由于multipart方式上传不够优雅

1、要不就是不可以实时获取到流,必须要等到全上传完毕 【MultipartFile 参数方式】

2、要不就是关闭框架中multipart【MultipartResolver】 解析的功能 【spring.servlet.multipart.enabled=false】,这样会导致MultipartFile 参数方式全部失败

并且获取到流中的body是按照multipart 方式处理的,有boundary边界等新,非原始的文件信息

更好的处理文件上传,直接使用binary方式,可以直接流中获取文件的原始内容

curl --limit-rate 100k  -X POST 'http://127.0.0.1:8080/upload' -H 'Content-Type: application/x-sql' --data-binary '@/Users/siqi/Desktop/file.sql'>out.txt

控制器示例

    @RequestMapping(value = "/upload")
    publicIdNamePair upload(HttpServletRequest request)
        throwsIOException {

//实时读取上传的文件信息 ServletInputStream inputStream =request.getInputStream(); StringBuffer sb = newStringBuffer(); ServletInputStream fis =inputStream; int b = 0; while ((b = fis.read()) != -1) { sb.appendCodePoint(b); } inputStream.close(); System.out.println(sb.toString()); IdNamePair idNamePair = newIdNamePair(); idNamePair.setId(123); idNamePair.setName("蓝银草"); returnidNamePair; }

php请求示例

<?php


$curl =curl_init();

curl_setopt_array($curl, array(
    CURLOPT_URL => "http://127.0.0.1:8080/upload",CURLOPT_RETURNTRANSFER => true,CURLOPT_ENCODING => "",CURLOPT_MAXREDIRS => 10,CURLOPT_TIMEOUT => 0,CURLOPT_FOLLOWLOCATION => true,CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,CURLOPT_CUSTOMREQUEST => "POST",CURLOPT_POSTFIELDS => file_get_contents("/Users/siqi/Desktop/file.sql"),CURLOPT_HTTPHEADER => array(
        "Content-Type: application/octet-stream",),));

$response = curl_exec($curl);

curl_close($curl);
echo $response;

参考:

https://stackoverflow.com/questions/16462088/php-curl-and-stream-forwarding

https://www.php.net/manual/zh/function.curl-setopt.php

https://www.ashleysheridan.co.uk/blog/Creating+a+Streaming+Proxy+with+PHP#stream-the-response

免责声明:文章转载自《接口中转stream传输 request/response》仅用于学习参考。如对内容有疑问,请及时联系本站处理。

上篇.Net 文件名后缀的详细解释椭圆曲线密码学在OpenSSL中的实现下篇

宿迁高防,2C2G15M,22元/月;香港BGP,2C5G5M,25元/月 雨云优惠码:MjYwNzM=

相关文章

gitlab 笔记

#http://www.jianshu.com/p/060e7223e211?open_source=weibo_search docker stop gitlabdocker stop redisdocker stop postgresqldocker rm gitlabdocker rm redisdocker rm postgresql docker...

【web端权限维持】利用ADS隐藏webshell

0X01 前言   未知攻,焉知防,在web端如何做手脚维护自己拿到的权限呢?首先要面临的是webshell查杀,那么通过利用ADS隐藏webshell,不失为一个好办法。 0X02 利用ADS隐藏webshell   关于ADS的介绍,就不再阐述,详见尾部参考资料。   PHP 一句话木马:<?php @eval($_POST['chopper']...

Redis使用

一、定义 redis是nosql产品之一,nosql就是无需编写复杂的sql语句。是Remote Dictionary Server(远程字典数据服务)的缩写。 由意大利人 antirez(Salvatore Sanfilippo)  开发的一款 内存高速缓存数据库。该软件使用C语言编写,它的数据模型为 key-value。 它支持丰富的数据结构(类型),...

数组中的filter函数,递归以及一些应用。

当我们用一个东西时候我们必须知道的是?why---where----how---when。一个东西我们为什么用?在哪用?怎么用?何时用?而不是被动的去接受一些东西。用在js里边我觉得也会试用。一直追求源生js,虽然也都背过好多东西,但是随着时间的流逝,工作的繁忙都忘了,有时甚至一点印象都没有,这让我开始思考我的学习方法了已经思维方式了。我们要记得不是简单的...

Android 文件的选择

Android 文件的选择 打开文件选择器 private void showFileChooser() { Intent intent = new Intent(Intent.ACTION_GET_CONTENT); intent.setType("*/*"); intent.addCat...

axios的基本使用

<!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title></title> <link rel="stylesheet" href="https://unpk...