Questions tagged [curl]

curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

cURL (/kə:(r)l/) is a computer software project providing a library and command-line tool for transferring data using various protocols. The cURL project produces two products, libcurl and cURL. It was first released in 1997. The name originally stood for "see URL".

see more @wikipedia and StackOverflow

740 questions
19
votes
5 answers

How to make a Curl POST call in Windows?

I am trying to issue the following command under Windows 10: D:\>curl -i -X POST -H "Content-Type:application/json" -d '{ "firstName" : "Frodo", "lastName" : "Baggins" }' http://localhost:8080/people Unfortunately, it produces numerous errors: curl:…
Dims
  • 13,672
16
votes
2 answers

How to get and use certificates with curl

Entering the following command curl -k email@mail.com:password https://www.example.com Returns the following error <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301…
Joshua Robison
  • 582
15
votes
2 answers

POSTing a file's contents with CURL

(The following is backstory, you can jump down to "Question:" if you'd like) There's a webpage for one of my courses that accepts input text (expected to be Python code) and then gives some tokenized response. It's meant to be used as a reference…
Cornstalks
  • 327
15
votes
1 answer

How do I remotely fetch files from redirected URLs from a terminal?

I want to fetch a tarball of this python library from terminal. https://github.com/simplegeo/python-oauth2/downloads However, I cannot simply call wget https://github.com/simplegeo/python-oauth2/tarball/master because doing so retrieves a web page.…
David Faux
  • 4,939
15
votes
1 answer

How do I pass question marks to server when newer `curl` (8.11.1) strips question mark where older (7.29.0) did not?

Using a device from mini-circuits that used to work just fine when queried with an older version of curl. As can be seen here many commands expect a question mark as part of the command. This worked with no issue in the past and I can still run with…
nhed
  • 466
14
votes
3 answers

Site displays with Curl but not within the browser

I've set up a dev-enviroment on Debian 7 and installed the following services: Apache Bind9 I have one site that is reachable through "asix.debian.local". I've set up the zone and added NS and A registry to the db.debian.local file. Everything is…
Borjante
  • 243
13
votes
2 answers

How can I instruct curl to reuse credentials after it followed a redirect?

I know that curl can follow redirects (the -L argument), and I know that I can provide credentials to a request ( -u username:password ), but I want to use both all the way down the redirect chain. How can I tell curl to reuse the username and…
karatedog
  • 867
13
votes
2 answers

How to convert HTML to text?

How it is possible to convert HTML to text file in Linux? For example I want to curl a query to Google, then convert the output html to text and read converted text on my terminal. I am using RHEL6.
rivu
  • 282
13
votes
1 answer

How to download parts of the same file from different sources with curl/wget?

I have a quite large file hosted on five different servers. I would like to be able to download different parts of the file from each server and subsequently concatenate the parts, in order to produce the original file. Is there any way to do this…
Molotoff
  • 318
12
votes
1 answer

Exit bash script when curl gets a non 200 HTTP status

I have a bash script setup to perform a few curl requests for currency in EUR INR JPY do curl -i --data '{"currency": "'$currency'"}' -H "Accept: application/json" -H "Content-Type: application/json" http://0.0.0.0:8080/price && echo done Is there a…
Marklar
  • 384
12
votes
2 answers

bash: source from URL

In addition to my own computer, I sometimes use an Ubuntu cluster at my school. Rather than manually keep my .bashrc's in sync, I would like to make the school cluster's .bashrc source my personal .bashrc from DropBox via a URL. However, when I…
AlcubierreDrive
  • 417
12
votes
4 answers

How "legal" is site-scraping using cURL?

Recently I was experimenting with the cURL, and I found lot is possible with it. I built a small script that crawls a musical site, which plays online songs. On the way of my experiment, I found that it is possible to crawl the song source also..…
Chetan Sharma
  • 487
11
votes
2 answers

How to download a private GitHub repo with no 'git' on my local machine?

What I want to do is to download private repository archive from GitHub, extract it, remove archive file and copy some directories that are inside downloaded project. I tried to use wget but I cannot authorize myself: wget --header='Authorization:…
Kamil Lelonek
  • 223
11
votes
1 answer

cURL Upload file and send POST data

I have a web server running some PHP that checks for an image ( curl -F 'imageName=@myimage' ) and it also checks the POST data for username=&password= . When the PHP checks _REQUEST I cannot just do: curl -F 'imageName=@myimage' \…
flumpb
  • 495
11
votes
2 answers

How do I send a digest auth request using curl?

While searching for a guide I found this example on Wikipedia GET /dir/index.html HTTP/1.0 Host: localhost Authorization: Digest username="Mufasa", realm="testrealm@host.com", nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093", uri="/dir/index.html",…
mina nageh
  • 572
1 2
3
…
49 50