杰瑞科技汇

Python 3.4中aiohttp如何使用?

Of course. Here is a comprehensive guide to using aiohttp with Python 3.4, including installation, basic usage, advanced patterns, and important considerations.

Python 3.4中aiohttp如何使用?-图1
(图片来源网络,侵删)

What is aiohttp?

aiohttp is a library for making asynchronous HTTP requests and building asynchronous web servers in Python. It is built on top of asyncio, the asynchronous I/O framework introduced in Python 3.4.

The key advantage is non-blocking I/O. While your program is waiting for a response from a server, it can use that time to send other requests or perform other tasks, making it extremely efficient for I/O-bound operations like web scraping or calling multiple APIs.


Installation

First, you need to install the library. Open your terminal or command prompt and run:

pip install aiohttp

The Basics: Making a GET Request

Let's start with the most common task: fetching data from a URL. The core of aiohttp is the ClientSession object. You should always use a with statement to ensure the session is properly closed.

Python 3.4中aiohttp如何使用?-图2
(图片来源网络,侵删)

Example: Fetching a single web page

# Ensure you have Python 3.4+ and aiohttp installed
# pip install aiohttp
import asyncio
import aiohttp
async def fetch_url(session, url):
    """
    Fetches a single URL using the provided session.
    """
    try:
        async with session.get(url) as response:
            # Check if the request was successful (status code 200-299)
            response.raise_for_status() 
            # .text() returns the response body as a string
            html = await response.text()
            print(f"Successfully fetched {url}")
            print(f"First 100 characters: {html[:100]}...")
    except aiohttp.ClientError as e:
        print(f"An error occurred while fetching {url}: {e}")
async def main():
    """
    The main entry point for the asyncio program.
    """
    target_url = 'http://python.org'
    # Create a ClientSession. It's best practice to reuse one.
    async with aiohttp.ClientSession() as session:
        await fetch_url(session, target_url)
# Python 3.4 requires the explicit event loop
if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()

Explanation:

  1. async def fetch_url(...): This function is a coroutine. The async keyword is mandatory for any function that uses await.
  2. async with aiohttp.ClientSession() as session:: This creates a session object. The async with block ensures that the session is closed automatically when the block is exited, which is crucial for releasing resources.
  3. async with session.get(url) as response:: This sends an HTTP GET request. The async with block here ensures the response connection is closed. The response object is an aiohttp.ClientResponse.
  4. response.raise_for_status(): A convenient method that raises an aiohttp.ClientError if the status code is 400 or higher (e.g., 404 Not Found, 500 Server Error).
  5. await response.text(): This is the key to asyncio. We are awaiting the result of the text() method. This means the fetch_url coroutine will pause here, and the event loop can run other coroutines (like fetching another URL) while it waits for the network response to arrive. The .text() method decodes the response body as UTF-8.

Advanced Pattern: Fetching Multiple URLs Concurrently

This is where aiohttp truly shines. Instead of fetching URLs one by one, we can fetch them all at the same time.

We'll use asyncio.gather() to run multiple coroutines concurrently.

import asyncio
import aiohttp
async def fetch_url(session, url):
    """
    Fetches a single URL and returns its URL and status.
    """
    try:
        async with session.get(url, timeout=10) as response:
            response.raise_for_status()
            # We don't need the full content for this example
            return url, response.status
    except asyncio.TimeoutError:
        return url, "TIMEOUT"
    except aiohttp.ClientError as e:
        return url, f"ERROR: {e}"
async def main():
    """
    Fetches multiple URLs concurrently.
    """
    urls = [
        'http://python.org',
        'http://pypi.org',
        'http://github.com',
        'http://non-existent-domain-12345.com', # This will fail
        'http://httpbin.org/delay/5' # This will take 5 seconds
    ]
    async with aiohttp.ClientSession() as session:
        # Create a list of tasks to be run concurrently
        tasks = [fetch_url(session, url) for url in urls]
        # asyncio.gather runs all tasks and waits for them all to complete
        results = await asyncio.gather(*tasks)
        print("\n--- Results ---")
        for url, status in results:
            print(f"{url:<40} | Status: {status}")
if __name__ == '__main__':
    # In Python 3.7+, you can just use asyncio.run(main())
    # For Python 3.4, we use the older loop method
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()

Explanation:

Python 3.4中aiohttp如何使用?-图3
(图片来源网络,侵删)
  1. tasks = [fetch_url(session, url) for url in urls]: This is a list comprehension that creates a list of coroutine objects, not their results.
  2. *`results = await asyncio.gather(tasks)`**: This is the magic.
    • asyncio.gather() takes multiple awaitable objects (our coroutines).
    • It schedules them all to run on the event loop concurrently.
    • The await keyword pauses the main coroutine until all the tasks passed to gather() are finished.
    • gather() then returns a list containing the results of each coroutine in the same order as the input tasks.

POST Requests and Sending Data

Making a POST request is just as easy. You use the session.post() method and pass your data to the data or json parameter.

Example: Sending JSON data

import asyncio
import aiohttp
import json
async def post_data(session):
    """
    Sends a POST request with JSON data.
    """
    url = 'http://httpbin.org/post'
    payload = {'key': 'value', 'number': 123}
    # The `json` parameter automatically sets the Content-Type header
    async with session.post(url, json=payload) as response:
        response.raise_for_status()
        response_data = await response.json() # .json() parses the response as JSON
        print(f"Successfully posted to {url}")
        print("Response JSON:")
        print(json.dumps(response_data, indent=2))
async def main():
    async with aiohttp.ClientSession() as session:
        await post_data(session)
if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()

Explanation:

  • session.post(url, json=payload): By passing a Python dictionary to the json parameter, aiohttp automatically serializes it to a JSON string and sets the Content-Type header to application/json. This is much cleaner than doing it manually.

Important Considerations for Python 3.4

While aiohttp works with Python 3.4, there are some key differences and limitations compared to modern Python (3.7+).

Running the Event Loop

  • Python 3.4: You must manually get and manage the event loop.
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(main_coroutine())
    finally:
        loop.close()
  • Python 3.7+: The asyncio.run() function handles this for you.
    asyncio.run(main_coroutine()) # Much simpler!

Context Managers (async with)

  • Python 3.4: async with is fully supported and is the recommended way to handle ClientSession and ClientResponse to ensure resources are cleaned up.
  • Python 3.10+: Introduced the contextlib.asynccontextmanager decorator, but async with remains the standard.

Error Handling

  • Python 3.4: The primary exception for client-side errors is aiohttp.ClientError. The response.raise_for_status() method is the best way to handle HTTP errors.
  • Python 3.10+: aiohttp introduced more specific exceptions like aiohttp.ClientResponseError, aiohttp.ClientPayloadError, etc., for more granular error handling.

aiohttp Version Compatibility

  • When using Python 3.4, you should be aware that the latest versions of aiohttp might drop support for it. It's best to check the aiohttp documentation or release notes to see which version is compatible. For Python 3.4, you might need to install an older version of aiohttp.

Summary

Feature Python 3.4 aiohttp Modern Python (asyncio.run())
Running Code loop = asyncio.get_event_loop()
loop.run_until_complete(my_coro())
asyncio.run(my_coro())
Session async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
Request async with session.get(url) as resp: async with session.get(url) as resp:
Awaiting Data data = await resp.text() data = await resp.text()
Error Handling try...except aiohttp.ClientError try...except aiohttp.ClientResponseError (more specific)

For any new project, it is highly recommended to use a modern version of Python (3.7 or newer) to take advantage of the simpler asyncio.run() syntax and other language improvements. However, if you are maintaining legacy code on Python 3.4, aiohttp is a powerful and effective tool for adding asynchronous capabilities.

分享:
扫描分享到社交APP
上一篇
下一篇