STM32 symbol generator

README.md

@@ -1,2 +1,114 @@
-## :warning: 301 Moved Permanently
-Location: https://gitlab.com/kicad/libraries/kicad-library-utils
+kicad-library-utils
+===============
+
+## / directory
+
+**check_lib_table.py**: Checks the validity of a library table against given libraries
+
+**download_pretty_libs.py**: Download or update KiCad version 4 footprint libraries
+
+## schlib directory
+
+**checklib.py**: Script for checking [KLC][] compliance of schematic symbol libraries.
+
+**schlib.py**: A Python module for parsing KiCad's schematic library file format.
+
+**test_schlib.sh**: A shell script used to validate the generation of files of the schlib module.
+
+**fix-pins.py**: A script created to help adapt existing library files to the [KiCad Library Convention][KLC] by testing some cases of x/y "wrong" pin positions and trying to fix them. The description of the cases are explained in the header of the script file.
+
+**move_part.py**: Script to move components between libraries.
+
+**autogen/**: Scripts for automatically generating schematic symbol libraries.
+
+## sch directory
+
+**sch.py**: A Python module for parsing KiCad's schematic file format.
+
+**test_sch.sh**: A shell script used to validate the generation of files of the sch module.
+
+**add_part_number.py**: This script is used to add/edit part number fields in the schematic files.
+
+**update_footprints.py**: This script updates the footprint fields of `.sch` files using a `.csv` file as input.
+
+## pcb directory
+
+**check_kicad_mod.py**: Script for checking [KLC][] compliance of footprint files.
+
+**kicad_mod.py**: A Python module for loading, editing, and saving KiCad footprint files.
+
+**check_3d_coverage.py**: Script for checking which KiCad footprints in a `.pretty` library have 3D models. It also shows unused 3D model files.
+
+[KLC]: http://kicad-pcb.org/libraries/klc/
+
+How to use
+==========
+
+## Schematic Library Checker
+
+    # first get into schlib directory
+    cd kicad-library-utils/schlib
+
+    # run the script passing the files to be checked
+    ./checklib.py path_to_lib1 path_to_lib2
+
+    # to check a specific component you can use the -c flag
+    ./checklib.py -c component_name path_to_lib1
+
+    # run the following 'h'elp command to see other options
+    ./checklib.py -h
+
+## Adding Part Number (PN) to Schematic Files
+
+    # first get into sch directory
+    cd kicad-library-utils/sch
+
+    # use the following command to add empty "MPN" fields in the schematic files
+    ./add_part_number.py path_to_sch1 path_to_sch2
+
+    # use the following command to add/edit the PN field using the passed csv file
+    # the default behaviour is search for "Reference(s)" and "MPN" columns in the csv
+    # the BOM generated by bom_csv_grouped_by_value plugin can be used after
+    # manually add the MPN field in the Collated Components section (for example)
+    ./add_part_number.py --bom-csv=path_to_bom_csv path_to_sch_files/*.sch
+
+    # run the following 'h'elp command to see other options
+    ./add_part_number.py -h
+
+
+## Footprint Checker
+
+    # first get into pcb directory
+    cd kicad-library-utils/pcb
+
+    # run the script passing the files to be checked
+    ./check_kicad_mod.py path_to_fp1.kicad_mod path_to_fp2.kicad_mod
+
+    # Add `-v`, `-vv`, or `-vvv` for extra verbose output. The most useful is `-vv`, which explains in details the violations. Ex: 
+    ./check_kicad_mod.py path_to_fp1.kicad_mod path_to_fp2.kicad_mod -vv
+
+    # run the following 'h'elp command to see other options
+    ./check_kicad_mod.py -h
+
+
+## 3D Coverage Checker
+
+    # first get into pcb directory
+    cd kicad-library-utils/pcb
+
+    # run the script to check all footprints
+    ./check_3d_coverage.py
+
+    # run the script to check only the specified .pretty folder
+    ./check_3d_coverage.py --prettty Housings_SOIC
+
+    # run the following 'h'elp command to see other options
+    ./check_3d_coverage.py -h
+
+
+Notice
+======
+
+The scripts use a different algorithm to generate files in relation to the KiCad saving action. That will result output files with more modified lines than expected, because the line generally are repositioned. However, the file still functional.
+
+Always check the generated files by opening them on KiCad. Additionally, if you are working over a git repository (if not, you should) you can commit your work before proceed with the scripts, this will put you safe of any trouble. Also, you would use git diff to give a look at the modifications.

schlib/autogen/stm32/.gitignore

@@ -0,0 +1,3 @@
+stm32_datasheets/
+db/
+mcu/

schlib/autogen/stm32/README.md

@@ -6,15 +6,18 @@ STM32 devices.
 ## Prerequisites
 
 * XML files, taken from STM32CubeMX install (from db/mcu folder).
-* Datasheet PDFs, downloaded from ST's website.  A current list as of
-  2018-02-06 can be found in datasheets.txt.  These PDFs can be downloaded all
-  at once with `wget -P datasheets -ci datasheets.txt`.
+* Datasheet PDFs, downloaded from ST's website.  The helper tool
+  `stm32_get_datasheets.py` will download all available STM32 datasheets into
+  `./stm32_datasheets/`.
 * [pdfminer](https://github.com/euske/pdfminer) tool.
 
 ## Running
 
 If you have the correct XML files, just run
 `./stm32_generator.py xmldir pdfdir`, where `xmldir` is a directory containing
 all the necessary XML files, and `pdfdir` is a directory containing all the PDF
-files.  Make sure the XML files have the correct format as there is no error
-checking.
+files (e.g. `./stm32_datasheets/`).  Make sure the XML files have the correct
+format as there is no error checking.
+
+Mining data from the pdf filestakes time, expect about 1-2 minutes per
+datasheet and CPU core.

schlib/autogen/stm32/stm32_generator.py

@@ -6,6 +6,8 @@
 import os
 import re
 import sys
+import multiprocessing
+from itertools import repeat
 
 from lxml import etree
 
@@ -344,9 +346,8 @@ def create_symbol(self, gen):
                 f"{{ram}}KB RAM, {freqstr}{voltstr}{self.io} GPIO, "
                 f"{pkgstr}")
         keywords = f"{self.core} {self.family} {self.line}"
-        datasheet = "" if self.pdf is None else (f"http://www.st.com/"
-                f"st-web-ui/static/active/en/resource/technical/document/"
-                f"datasheet/{self.pdf}")
+        datasheet = "" if self.pdf is None else (f"https://www.st.com/"
+                f"resource/en/datasheet/{self.pdf}")
 
         # Make the symbol
         self.symbol = gen.addSymbol(self.name, dcm_options={
@@ -670,6 +671,13 @@ def draw_symbol(self):
                 visibility=SymbolField.FieldVisibility.INVISIBLE)
 
 
+def run_pdf2txt(pdffile, pdfdir):
+    pdffile = os.path.join(pdfdir, pdffile)
+    pdfparsedfile = pdffile + ".par"
+    if not os.path.isfile(pdfparsedfile) and pdffile.endswith(".pdf"):
+        logging.info(f"Converting: {pdffile}")
+        os.system("pdf2txt.py -o " + pdfparsedfile + " " + pdffile)
+
 def main():
     parser = argparse.ArgumentParser(
             description='Generator for STM32 microcontroller symbols')
@@ -696,28 +704,25 @@ def main():
 
     # Parse text from PDFs
     for _, _, filenames in os.walk(args.pdfdir):
-        for pdffile in filenames:
-            pdffile = os.path.join(args.pdfdir, pdffile)
-            pdfparsedfile = pdffile + ".par"
-            if not os.path.isfile(pdfparsedfile) and pdffile.endswith(".pdf"):
-                logging.info(f"Converting: {pdffile}")
-                os.system("pdf2txt.py -o " + pdfparsedfile + " " + pdffile)
-        break
+        filenames.sort()
+        with multiprocessing.Pool() as pool:
+            pool.starmap(run_pdf2txt, zip(filenames, repeat(args.pdfdir)))
 
     # Load devices from XML, sorted by family
     libraries = {}
     for _, _, filenames in os.walk(args.xmldir):
         filenames.sort()
         for xmlfile in filenames:
-            # Load information about the part(s)
-            mcu = Device(os.path.join(args.xmldir, xmlfile), args.pdfdir)
-            # If there isn't a SymbolGenerator for this family yet, make one
-            if mcu.family not in libraries:
-                libraries[mcu.family] = SymbolGenerator(
-                        lib_name=f"MCU_ST_{mcu.family}")
-            # If the part has a datasheet PDF, make a symbol for it
-            if mcu.pdf is not None:
-                mcu.create_symbol(libraries[mcu.family])
+            if xmlfile.startswith("STM"):
+                # Load information about the part(s)
+                mcu = Device(os.path.join(args.xmldir, xmlfile), args.pdfdir)
+                # If there isn't a SymbolGenerator for this family yet, make one
+                if mcu.family not in libraries:
+                    libraries[mcu.family] = SymbolGenerator(
+                            lib_name=f"MCU_ST_{mcu.family}")
+                # If the part has a datasheet PDF, make a symbol for it
+                if mcu.pdf is not None:
+                    mcu.create_symbol(libraries[mcu.family])
         break
 
     # Write libraries