Copyright 2002-2024 Free Software Foundation, Inc.

  Copying and distribution of this file, with or without modification,
  are permitted in any medium without royalty provided the copyright
  notice and this notice are preserved.

The development sources for GNU Texinfo are available through git
at Savannah:
  https://savannah.gnu.org/git/?group=texinfo

See also files:
tp/README, tp/t/README, tp/tests/README, TODO, TODO.HTML

Development tools
-----------------

This distribution uses whatever versions of Automake, Autoconf, and
Gettext are listed in NEWS; usually the latest official releases.  If
you are getting the sources from the development repository (or change
configure.ac), you'll need to have these tools installed to (re)build.
You'll also need help2man (for building man pages) and gperf (for building
"unicase/*" and "uninorm/*" modules from gnulib).  If you modify
texindex/ti.twjr, you'll need gawk >= 4.0.  All of these programs are
available from https://ftp.gnu.org/gnu.  You'll also need pod2man
for building a man page, which should be part of the Perl core
distribution.

After getting the development sources, and installing the tools above,
you can run
 ./autogen.sh
and then, for example,
 ./configure -C CFLAGS='-g' PERL_EXT_CFLAGS='-g'
and then
 make

The -C tells configure to cache test results, which usually speeds
things up a bit.

After the initial autogen && configure, simply running make should suffice.

Gettext or help2man not installed do not cause configure to fail,
though configure shows if they were found.  This is because a release
does not require those tools.  Indeed, both prerequisites and
result files are shipped in a release, such that the tools are only
needed if the prerequisite changed.  The tools are needed, however, when
building from development sources, as result files are not under version
control.  Make will fail with an explicit "missing command" for help2man,
and with a "command not found" error for a Gettext utility command.

Running make in one particular subdirectory is possible, for example
make -C info.  However there are interdependencies between the 
subdirectories, notably on gnulib, so if you don't want to run "make", 
you may have to run "make -C gnulib/lib" first.

Additionally, make dist may not work until make has been run at least 
once, because of rules to create man pages under the man/ directory.

"make dist" will fail if the use of Perl XS extension modules is 
disabled and there is no Makefile in the XS subdirectory.


Using git
---------

This section is if you have write access to the git repository.

Usually commits to the git repository should include a ChangeLog
entry.  Please follow the existing style (the GNU Coding Standards
has a guide).

You can automatically use the contents of the most recent ChangeLog
entry with a git commit hook .git/hooks/prepare-commit-msg

------------------------------------
#!/bin/sh

# $1 - file that contains commit log message
# $2 - source of commit message

outfile="$1"

case $2 in
  message|template|merge|squash|commit)
    ;;
  *)
   # Use latest ChangeLog entry as commit message
   sed -n -e '1,/^\w*$/d' -e '/^[^	]/q' -e '{s/^	//;p}' ChangeLog  >"$outfile"
;;
esac
------------------------------------

If you are frequently applying patches sent by other people, the following
may be useful to have in .git/hooks/post-commit to get the --author
information from the ChangeLog:

------------------------------------
#!/bin/sh

# post-commit

# avoid recursion
test $IN_GIT_HOOK_POST_COMMIT && exit 0
export IN_GIT_HOOK_POST_COMMIT=1

# Get author from latest ChangeLog entry.  Remove date, leading and trailing
# spaces, and condense multiple spaces to 1.
CL_author=$(head -n 1 ChangeLog | sed -e 's/^[[:digit:]-]*//' -e 's/^ *//' -e 's/ *$//' -e 's/  \+/ /')

# Get author from last commit
HEAD_author=$(git log -1 --pretty=format:'%an <%ae>')

if test "$CL_author" != "$HEAD_author" ; then
    git commit --amend --author="$CL_author" --reuse-message=HEAD

    echo "post-commit: changed author from '$HEAD_author'"
    echo "post-commit: to '$CL_author'."
    echo "post-commit: to override, run:"
    echo "post-commit: IN_GIT_HOOK_POST_COMMIT=1 git commit --amend -C HEAD --author=\"$HEAD_author\""
fi
------------------------------------

Please ensure they have appropriate copyright assignments in place with
the FSF before committing other people's changes, and check with a
maintainer if unsure.

When unable to push commits due to other commits being made, please
use "git pull --rebase" (the default for "git pull" complicates the
git history).  To deal with conflicts in the ChangeLog, you should
install the git-merge-changelog program.

You can get better output from "git diff" for Texinfo files by putting
the following section in your .gitconfig file:

[diff "texinfo"]
        xfuncname = "^(@node .*)$"

This shows which node each change occurred in.


Gnulib
------

This distribution uses Gnulib (http://www.gnu.org/software/gnulib)
to share common files.  Gnulib files used in Texinfo are checked in to
the repository.  A Gnulib directory is setup in two locations, in
the main directory and in tp/Texinfo/XS/.

To update the gnulib files, get a checkout of gnulib in a separate
directory, then run, say

  ../gnulib/gnulib-tool --add-import

in your top-level Texinfo directory and

  ../../../../gnulib/gnulib-tool --add-import

under tp/Texinfo/XS.  (gnulib-tool is in the gnulib source tree.)

The currently-used gnulib modules and other gnulib information are
recorded in gnulib/m4/gnulib-cache.m4.

gnulib-tool --add-import may also be used to add another gnulib module:
  ../gnulib/gnulib-tool --add-import other_gnulib_module

After running gnulib-tool --add-import or otherwise adding modules, it is
necessary to check what files were added or removed (e.g., run "git
status -u") and add new files to the repository with "git add".
Add any new generated files (typically gnulib/lib/foo.h from foo.h.in) 
to the ignore list in .gitignore.

When adding new modules, or sometimes even just when upgrading gnulib,
it may be necessary to update the LDADD variables in info/Makefile.am or
install-info/Makefile.am.  Check for any unfamiliar Makefile variables in
the output of "gnulib-tool --add-import", and work out which program
needs them.  You can look at the modules/* files in the gnulib checkout
to help work out which gnulib module is demanding which variable.

For the gnulib checkout in the XS directory, it is just
tp/Texinfo/XS/Makefile.am that may need to be updated.



Subdirectories in repository
----------------------------

In addition to the subdirectories listed in README, there is the 
following directory in the source control repository:

infog/ - HTML-Info reader using WebKitGTK library

Finally, the contrib/ directory contains additional files from users
provided for your reading and/or hacking pleasure.  They aren't part of
Texinfo proper or maintained by the Texinfo developers.




About running the Texinfo programs from a development source tree:

- Once the distribution is built, you can run the compiled programs
(info, install-info) out of the build tree without special settings; 
they don't try to read any installed data files.

- The texi2dvi script and texinfo.tex can be run as-is, since they 
are standalone and don't require compilation.  For the same reasons,
they are officially updated between full Texinfo releases, at
http://ftpmirror.gnu.org/texinfo.

- Regarding texi2any (aka makeinfo), you can run tp/texi2any.pl
directly.  This is the original source file for the program, so it's
convenient to be able to make changes and then run it.

To run the output "tp/texi2any" instead, you can set the environment
variable TEXINFO_DEV_SOURCE to 1.  Otherwise, it will try to use
Texinfo's Perl modules in the installed locations.  "tp/texi2any" uses
the Perl interpreter found by configure, so you might want to run that 
instead of texi2any.pl if it's different to the default interpreter in 
your environment.

To directly run programs out of source, you should set the top_builddir
variable to the top build directory, in order to have compiled modules
and translated in-document strings found.  If you use scripts with
names ending in .pl or .t test files, the source directory obtained
from the script name is used to determine the source directories and
you do not need to set them explicitely.  Otherwise, you may need to
set the srcdir and/or the top_srcdir variables.



References for working on various parts of the system:

If you want to delve into making a new backend for the Perl makeinfo,
the documentation in tp/Texinfo/Convert/Converter.pm is a good starting
point, as it describes the existing backends and other places to look.

If you want to delve into texinfo.tex, a thorough plain TeX reference
is available under the GFDL:
  TeX by Topic - http://www.eijkhout.net/texbytopic/texbytopic.html
Another book on plain TeX, also available under the GFDL, is a GNU package:
  TeX for the Impatient - http://www.gnu.org/software/teximpatient/
Occasionally you may need to know about the details of the PDF format.
A reference for this is the PDF reference, Sixth Edition, version 1.7,  
downloadable at http://www.adobe.com/devnet/pdf/pdf_reference_archive.html

The texindex program is implemented using the TexiWebJR literate
programming system, combining Texinfo and Awk
(https://github.com/arnoldrobbins/texiwebjr).  Running "make ti.pdf"
in the texindex/ subdirectory creates the printable form of the
program.  All the usual Texinfo output formats are possible.



- Updating copyright years in files

We try to handle updating years in copyright notices automatically to
relive people of the burden of remembering to update copyright years when
they edit a file for the first time in a new year.

In a new year, run "make grand-replace" from the top level.  Check changes
with "git diff" before committing.

Look for omissions by running, e.g.

find . \
  -name autom4te.cache -prune -o \
  -name .git -prune -o \
  -wholename ./tp/maintain/lib -prune -o \
  -name test -prune -o \
  -name "test-infodir" -prune -o \
  -name "gnulib" -prune -o \
  -type f \
  \( -name '*' \) \
  -not -name 'Makefile.in' \
  -not -name Makefile \
  -not -name "*.m4" \
  -not -name configure \
  -not -name "config.*" \
  -not -name "*~" \
  -exec perl -wnl -e '/20\d[^4] Free/ && print "$ARGV:$_"' '{}' \;

at the top level.  It's likely that the script doesn't find all the
years that should be updated yet.


- Note on "reproducible builds"

Generation of Texinfo releases has not been checked to be fully
reproducible (see https://reproducible-builds.org/).

However, in the interests of a reproducible distribution, GNU tar is
used with certain flags to generate a tar archive file containing input
for texi2any tests (by maintain/regenerate_non_ascii_tar_file).  GNU tar
will only be used with these flags if it is detected that it is available.
If other tar programs are used, this archive file will be not bit-for-bit
identical.



- When close to making a release:

NYTProf profiling for Perl code
* e.g. 'perl -d:NYTProf ../tp/texi2any.pl FILE.texi'.  See Devel::NYTProf
  man page.
callgrind profiling for XS code
* e.g. 'LD_BIND_NOW=1 valgrind --tool=callgrind perl ../tp/texi2any.pl \
  FILE.texi' then 'kcachegrind callgrind.out.*'.

run all tests with valgrind:
* in info/t/Init-test.inc, put valgrind in $ginfo, then check t/*.val.log
  files after running test suite.  Note that a small number of tests may
  report leaks unless info is compiled with CFLAGS='-O0'.
* edit install-info/tests/defs.in, uncomment valgrind line and run
  "./config.status install-info/tests/defs" to regenerate defs.
  Note that many tests may report leaks unless install-info is compiled
  with CFLAGS='-O0'.

* XS modules memory leak check checking with valgrind
  Add line "use Perl::Destruct::Level level => 1;" to texi2any.pl
  when running 'valgrind --leak-check=full' to make perl clean it
  its allocations.  Run e.g.
      valgrind --suppressions=texi2any.supp --log-file=val.log \
               --leak-check=full \
        perl ../tp/texi2any.pl ../doc/info-stnd.texi
  and other input files.

  Add line "use Perl::Destruct::Level level => 1;" to t/test_utils.pl
  Check tests with e.g.
    for f in t/*.t ; do valgrind --suppressions=texi2any.supp \
                                 --log-file=val.log-${f#t/} \
                                 --leak-check=full \
          perl -w $f ; done
 (could take 1-2 hours to finish).  check log files one by one afterwards.

Consider doing make check with autovivification turned off to find bugs
and make sure that it is possible to turn off autovivification during
developpement.  It may be interesting to set TEXINFO_XS=omit for this
verification.  The module turning off autovivification is not in Perl
core, so when commiting the lines turning autovivification off should
always be commented out.
To turn off autovivification, you can use:
 find tp/Texinfo tp/ext tp/init -type f -name '*.pm' -execdir perl -wpli -e 's/^#no autovivification qw\(fetch delete exists store strict\);/no autovivification qw\(fetch delete exists store strict\);/' '{}' ';'
To get back to the default:
 find tp/Texinfo tp/ext tp/init -type f -name '*.pm' -execdir perl -wpli -e 's/^no autovivification qw\(fetch delete exists store strict\);/#no autovivification qw\(fetch delete exists store strict\);/' '{}' ';'

Manual testing:
. process doc/texinfo-tex-test.texi with TeX and check that output is good.
. process some manuals with INFO_JS_DIR and check the JS interface is
  not broken.
. if necessary, check if the HTML customization API can be updated for
  - lilypond Documentation/lilypond.init
  - ffmpeg doc/t2h.pm

check for C compiler warnings by configuring with

our_CFLAGS='-Wall -Wformat-security -Wstrict-prototypes -Wno-parentheses'
./configure "CFLAGS=$our_CFLAGS" "PERL_EXT_CFLAGS=$our_CFLAGS"
unset our_CFLAGS

Not all compiler warnings have to be fixed, though.

check if po_document/adjust_translations.pl script is still needed (check
progress of translations at
<https://translationproject.org/domain/texinfo_document.html>, and check
whether script is working as intended.)

check at latest automake/autoconf/gettext/help2man version:
  . after installing latest version, run
      AUTOPOINT=true autoreconf -f -v -i
    followed by reverting unwanted changes
  . to upgrade gettext, run
      gettextize --no-changelog -f --po-dir=po --po-dir=po_document --dry-run
      gettextize --no-changelog -f --po-dir=po --po-dir=po_document
    after installing new version of gettext.
    if gettextize makes too many changes that need to be reverted, try
      autopoint -f --dry-run
      autopoint -f
    run "git status -u" to check for new files.  revert or remove any
    *.m4 files that downgrade from a newer serial number (from gnulib)
    to an older serial number (from gettext).  check for duplicate files
    under build-aux/m4 and gnulib/m4.

    see http://gnu.org/s/gnulib/manual/html_node/gettextize-and-autopoint.html
    for some hints.

  . also upgrade gettext under tp/Texinfo/XS.  run:
      gettextize -f --po-dir=../../../po --no-changelog
     and then revert any unwanted changes and remove any unwanted new files.

  . After upgrading automake/autoconf/gettext, run ./autogen.sh
    and/or "autoreconf --verbose --force --install" to update ancilliary
    files in build-aux and elsewhere.  Check changes before committing.

  . help2man upgrade -
      make -C man maintainer-clean ; ./config.status man/Makefile ; make -C man
  . mention new versions in NEWS

Update gnulib:
  # Under the top level, and also under tp/Texinfo/XS, which uses
  # a separate gnulib import.
  gnulib-tool --add-import
  'git status -u' and add untracked files

Use util/srclist-txi for checking files to be copied from gnulib

Update ppport.h in tp/Texinfo/XS.  Update Devel::PPPort via cpan to newest
available version, then run:
  perl -MDevel::PPPort -e'Devel::PPPort::WriteFile'
Check that this results in a newer version.

Check what changes ppport.h reports:
  perl ppport.h --compat-version=5.8.1 *.xs */*.xs main/get_perl_info.* main/build_perl_info.* main/api_to_perl.* main/call_perl_function.* convert/call_html_perl_function.* convert/build_html_perl_info.* convert/build_html_perl_state.* convert/get_converter_perl_info.* convert/get_html_perl_info.*

make po-check             # update po/POTFILES.in as needed
make po_document-check    # update po_document/POTFILES.in as needed

check indices of Texinfo manuals and check for duplicates (with <1> in Info)

Check "make ccheck" and "make vcheck" work in "doc/refcard".

Check mentions of configure flags, environment variables etc. in INSTALL are
up to date

Consider running tests of texi2dvi under contrib/nontests

- Official releases only:
make V=1 pdf and fix underfull/overfull boxes.

-------------------------------------------------------------------------
- Distribution checking

In addition to "make distcheck", there are other build system checks
which you can run manually.

# 'make dist' without 'make'
make distclean
./configure
make dist

# test make after maintainer-clean
./configure
make maintainer-clean
./configure
make

# test distcheck after maintainer-clean
./configure
make maintainer-clean
./configure
make distcheck

# test distcheck in out-of-source build after maintainer-clean
./autogen.sh
./configure
make maintainer-clean
rm -rf build
mkdir build
cd build
../configure
make distcheck

# test that distributed files are not generated in builddir after
# maintainer-clean (make distcheck only checks this for distclean).
./configure
make maintainer-clean
rm -rf build
mkdir build
cd build
../configure
make
make distclean
find . -type f


-------------------------------------------------------------------------
- Final steps for making a release

(Keep a copy of README-hacking open in a text editor to update these
release instructions.)

Check on correct branch (master or release/*)

git switch release/7.1
# remove stale generated headers from gnulib
git status -u | grep '\.h$' | xargs rm

* For release/7.1 branch only: need to "rm -f doc/refcard/Makefile.in"
after switching to prevent manually written Makefile being clobbered.
[Remove this note once release/7.1 no longer relevant.]

Then rebuild:

./autogen.sh ; ./configure ; make

Have a look at the output of "git status -u" to check for files that
  should be tracked in git or ignored.  Check for leftover result
  files under tp/tests which should be removed.

check "git stash list" for any work in progress

Check if "make tex-html-checks" results have been updated under tp/tests.

check OpenCSW build reports at
  https://buildfarm.opencsw.org/buildbot/waterfall?category=texinfo
(for master branch only)

Check that TEXINFO_DTD_VERSION has been updated to the next version in
  configure.ac if the DTD has been modified since the last release.
  See comments in configure.ac, and run (at the top level) make dtd-check.

Check "dist-xz" is in the option list in configure.ac (often removed
for speed when testing).

Update version number:
  update version number in configure.ac, util/texi2dvi, util/texi2pdf,
    js/info.js
  version number in txirefcard.tex (offical releases only)
  (cd texindex ; rm texindex.awk ; make)
  (cd tp && ./maintain/change_perl_modules_version.sh auto)
    -- this updates all the version numbers in the Perl modules and
       in tp/Texinfo/XS/configure.ac
check up to date copyright years in files relevant to --version calls
(tp/texi2any.pl, info/info.c, install-info/install-info.c, texindex/ti.twjr,
Pod-Simple-Texinfo/pod2texi)

Ensure texinfo.tex, htmlxref.cnf are updated on ftp.gnu.org.
Upload texi2dvi, texi2pdf (official releases only)

* For release/* branch only: propagate htmlxref.cnf from master

(cd tp ; maintain/regenerate_file_lists.pl) # list all test results

Ensure TXI_XLATE in doc/Makefile.am matches actual file list.

Update translations:
  rsync  -Lrtzv  translationproject.org::tp/latest/texinfo/ po
  rsync  -Lrtzv  translationproject.org::tp/latest/texinfo_document/ \
                 po_document # note the trailing slashes in these commands
  git status -u  # to check for new translations

run "make -C po check-linguas" and "make -C po_document check-linguas"
to check that LINGUAS under po and po_document match actual file list.

  make
  make update-po
  ( cd po_document ; for f in *.po ; do ./adjust-translations.pl $f ; done )
  ( cd po_document ; make update-po ) # in case po files change again after
                                      # last step


- Official releases only:
version and date in NEWS.
(cd tp && maintain/regenerate_documentlanguages-loc.pl)
  -- regenerates tp/Texinfo/Documentlanguages.pm (requires Text::CSV)

notice in ChangeLog.
one last "git diff" to check release commit looks good
make distcheck
(export MALLOC_CHECK_=2; make distcheck)  # repeat until clean
git commit and push

(If on a release branch, copy the NEWS message to NEWS on master)

-------------------------------------------------------------------------

- To do the actual upload:
pkg=texinfo
ver=7.0

then do one of:
gnupload --to alpha.gnu.org:$pkg $pkg-$ver.tar.xz                #pretest
gnupload --to ftp.gnu.org:$pkg   $pkg-$ver.tar.{gz,xz}           #official
gnupload --replace --to ftp.gnu.org:texinfo texi2dvi             #official
gnupload --replace --to ftp.gnu.org:texinfo texi2pdf             #official
   (Use --user option if not using default key)

pretest announcement -> bug-texinfo / beebe / platform-testers to try.
  bcc coordinator@translationproject.org.

For official releases: send announcement to info-gnu,
  cc bug-texinfo and bcc coordinator@translationproject.org.
news item at savannah.

#  Official releases only: tag source tree
git tag texinfo-6.6
git push --tags

#  ... set up dtd directory on web pages:
cd CVS_WEB/dtd # or wherever webpages checkout is
mkdir $ver && cvs add $ver
cp $tutil/texinfo.dtd $ver
cvs add -kb $ver/texinfo.dtd
cvs commit -m$ver $ver

-------------------------------------------------------------------------


- When official release is out there ...

Make sure texi2any and texinfo.tex are both installed, then build
web documentation with
  make -C doc wwwdoc-build

Copy documentation files to web checkout with, e.g.
  make -C doc \
  wwwdoc-install www_target=../../CVS_WEB/manual/
Check for removed files with, e.g. ls -ltu $(www_target)/*/html_node, 
and looking at file modification dates for files that have not
been replaced, followed by cvs rm -f.  Check for added files with
"cvs -qn update | less", followed by cvs add for lines beginning "?".
When done, run cvs commit.

update date and version number on home page (texinfo.html) and commit
as needed.

-------------------------------------------------------------------------

#  Official releases only: Contact root@tug.org to update texinfo at tug.org.
# If root@tug.org doesn't reply, can try webmaster@tug.org,
# or (last resort) board@tug.org.


#  ... post-release, or when development resumes:
configure.ac, util/texi2dvi: add "dev" to versions for clarity,
until it's time to do pretests again.