Profile: Dionysus

{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# -*- coding: utf-8 -*-\n", "\"\"\"\n", "Created on Tue Jan 12 19:40:03 2021" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "@author: Usuario\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Para instalar la libreria BeautifulSoup, abrimos Anaconda prompt y escribimos:
\n", " pip install beautifulsoup4" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Importamos las librerias necesarias" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from bs4 import BeautifulSoup\n", "from urllib.request import urlopen" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Crear un objeto BeautifulSoup" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Podemos acceder a la url para ver como es la web que queremos scrapear" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "url = \"http://olympus.realpython.org/profiles/dionysus\"\n", "# Abrimos la URL\n", "page = urlopen(url)\n", "# Leemos y decodificamos\n", "html = page.read().decode(\"utf-8\")\n", "# Crear objeto BeautifulSoup\n", "soup = BeautifulSoup(html, \"html.parser\")" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", "\n", "Profile: Dionysus\n", "\n", "\n", "\n", "

\n", "

\n", "

Name: Dionysus

\n", "

\n", "Hometown: Mount Olympus\n", "

\n", "Favorite animal: Leopard
\n", "
\n", "Favorite Color: Wine\n", "\n", "\n", "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "soup" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " CUIDADO :
\n", "Puede que la variable soup, no aparezca en el explorador.
\n", "Esquina superior derecha -- Desmarcar: Excluir objetos llamables y modulos" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Utilizando el objeto BeautifulSoup creado ####" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Mediante los metodos contenidos dentro de los objetos BeautifulSoup,
\n", "se pueden llevar a cabo diferentes acciones" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Extraemos todo el texto eliminando las etiquetas HTML" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "url http://olympus.realpython.org/profiles/dionysus\n" ] } ], "source": [ "print (\"url\"+url}\")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "Profile: Dionysus\n", "\n", "\n", "\n", "\n", "\n", "Name: Dionysus\n", "\n", "Hometown: Mount Olympus\n", "\n", "Favorite animal: Leopard \n", "\n", "Favorite Color: Wine\n", "\n", "\n", "\n", "\n" ] } ], "source": [ "print(soup.get_text())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Podriamos eliminar las lineas en blanco" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Profile: DionysusName: DionysusHometown: Mount OlympusFavorite animal: Leopard Favorite Color: Wine\n" ] } ], "source": [ "noblanklines=soup.get_text().replace(\"\\n\",\"\")\n", "print(noblanklines)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "O realizar busquedas en el texto" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "109" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "findtext=soup.get_text().find('Wine')\n", "findtext # Devuelve la posicion de la primera ocurrencia" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A veces interesa mantener las etiquetas HTML para poder realizar
\n", "busquedas de elementos especifios, como imagenes" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[

]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "soup.find_all(\"img\")\n", "# Devuelve una lista con los elementos contenidos en esa etiqueta" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Podemos extraer el contenido de cada etiqueta en una variable" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "image1, image2 = soup.find_all(\"img\")\n", "# Dentro de las variables creadas, podemos observar los metodos o propiedades" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Podemos pedir que nos diga el tipo de eqtiqueta HTML del objeto
\n", "con la propiedad .name" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'img'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "image1.name" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Se podria acceder a los atributos HTML de las etiquetas
\n", "especificando su nombre entre corchetes. (Como en un diccionario)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Etiqueta con un solo atributo
\n", "