Jim Tcl version 0.77

ANNOUNCE: Jim Tcl version 0.77

Jim Tcl 0.77 has been released and is available from:

http://repo.or.cz/w/jimtcl.git or https://github.com/msteveb/jimtcl

Find out all about Jim Tcl at http://jim.tcl.tk/

CHANGES SINCE VERSION 0.76

This release contains some bug fixes plus a number of additional features. A summary is below. See git for the full changelog.

Thanks to everyone who contributed to this release.

Bugs fixed in version 0.77

  • exec - better handline of pipeline abnormal termination
  • exec - fix append redirection on Windows
  • regsub - fix substitution with trailing backslash
  • expr - improved mathfunc handling and pow/**
  • chained tailcalls were not always being run

Features added in version 0.77

  • Add support for configure --docdir=...
  • Add support for jimsh --help
  • Add support for booleans in string is and expressions (true, false, on, off, yes, no)
  • aio - add sync, openssl bindings, posix locking
  • expr - add support for atan2, hypot and fmod
  • regexp, regsub:
    • Add support for \D, \W and \S
    • Add partial support for \A, \Z
    • Add support for all character classes: [[::blank:]], [[::xdigit::]], etc.
  • Update included sqlite3 to v3.14.1
  • Add $tcl_platform(engine)
  • Add basic (optional) zlib support
  • Add interp child interprester support
  • oo:
    • Add support for constructor, runs on new object creation
    • Add support for unknown method
  • Add Travis and AppVeyor continuous integration support
  • Use pkg-config in configure to find packages if possibe

Steve Bennett (steveb@workware.net.au)

Comments >>>

Jim Tcl version 0.76

ANNOUNCE: Jim Tcl version 0.76

Jim Tcl 0.76 has been released and is available from:

http://repo.or.cz/w/jimtcl.git or https://github.com/msteveb/jimtcl

Find out all about Jim Tcl at http://jim.tcl.tk/

CHANGES SINCE VERSION 0.75

This release contains some bug fixes plus a number of additional features. A summary is below. See git for the full changelog.

Thanks to everyone who contributed to this release.

Bugs fixed in version 0.76

  • file - fix stat file size for large files
  • array - avoid crash on unset variable
  • exec, file` - set umask before mkstemp
  • file copy - use binary mode when reading and writing files (Windows)
  • glob - work when intermediate dirs are not readable, and other fixes
  • aio - fix conflict between onexception and writable
  • namespace - restore namespace import support
  • alias - don’t rely on internal list never shimmering
  • Fix various minor problems reported by coverity
  • Fix info nameofexecutable after cd

Features added in version 0.76

  • exec, file - respect $TMPDIR
  • aio - optional argument addrvar for accept.
  • string - implement string cat (TIP #429)
  • file - add support for file link
  • info - add support for info source ?filename line?
  • Better script validation (e.g. missing trailing brackets)
  • Install tcltest compatibility package

Compatiblity

  • Some invalid scripts will now throw an error rather than silently executing
  • glob is more Tcl-compatible with respect to -directory and -tails, so may affect some scripts
  • The output of errorInfo has changed from "Runtime Error: <file>:<line>: ..." to "<file>:<line>: Error: ..."

Steve Bennett (steveb@workware.net.au)

Comments >>>

Jim Tcl version 0.75

ANNOUNCE: Jim Tcl version 0.75

Jim Tcl 0.75 has been released and is available from:

http://repo.or.cz/w/jimtcl.git or https://github.com/msteveb/jimtcl

Find out all about Jim Tcl at http://jim.tcl.tk/

CHANGES SINCE VERSION 0.74

This release contains some bug fixes plus a number of additional features. A summary is below. See git for the full changelog.

Thanks to everyone who contributed to this release.

Bugs fixed in version 0.75

  • eventloop - Correct file handler should be deleted
  • eventloop - Fix memory management of aio event handlers
  • local - Requires at least one argument
  • exec - Support nulls in exec immediate redirection
  • binary - More compliant with Tcl
  • aio - Fix aio recvfrom for non-inet socket
  • Fix string tolower buffer overflow for non-utf8
  • jimsh - consider scripts with trailing backslash as unfinished
  • Properly respect backslashes in comments
  • Avoid infinite recursion with namespace import
  • Fix dict/list shimmering with embedded nulls
  • Fix aio close from non-global namespace
  • tailcall should resolve command in current namespace
  • errors caught by catch shouldn`t affect later stacktrace
  • tailcall: properly merge tailcall frames
  • exec: fix reaping of detached processes
  • clock: handle case of format string too long
  • format: fix format %hd on some platforms
  • jim.c: fix some dict/list shimmering issues

Features added in version 0.75

  • binary, pack and unpack now support floating point
  • file copy -force handles source and target as the same file
  • format now supports %b for binary conversion
  • lsort now supports -unique and -real
  • Add support for half-close with aio close ?r|w?
  • Add socket pair for a bidirectional pipe
  • Add –random-hash to randomise hash tables for greater security
  • dict now supports for, values, incr, append, lappend, update, info and replace
  • file stat no longer requires the variable name

Compatiblity

  • The deprecated case command has been removed

Steve Bennett (steveb@workware.net.au)

Comments >>>

Binary Cheatsheet

I have been using Tcl for years, but I still need to lookup the manual every time I use binary format and binary scan. If you are like me, you will find this cheatsheet a handy reference.

Endian      
Little Big Host Type Notes
a     byte string format pads with nulls
A     byte string format pads with spaces, scan strips trailing spaces/nulls
b B   binary digits  
h H   hex digits  
c     8 bit integer list In other words, ASCII chars
s S t 16 bit integer list  
i I n 32 bit integer list  
w W m 64 bit integer list  
r R f single prec. float list  
q Q d double prec. float list  
x     null bytes count is number of bytes
X     move cursor back count is number of bytes to back up
@     move cursor to location count is byte position, zero based

Examples

set bin [binary format I2sH2a* {0x123 0x456} 17 ab "testing"]

# This returns 4 (number of conversions)
binary scan $bin I2sH2a* intlist intval hexval str

Steve Bennett (steveb@workware.net.au)

Comments >>>

Jim Tcl version 0.74

ANNOUNCE: Jim Tcl version 0.74

Jim Tcl 0.74 has been released and is available from:

http://repo.or.cz/w/jimtcl.git or https://github.com/msteveb/jimtcl

Find out all about Jim Tcl at http://jim.tcl.tk/

CHANGES SINCE VERSION 0.73

This release is mostly a bug fix release with some performance enhancements and a few small additional features. A summary is below. See git for the full changelog.

Thanks to everyone who contributed to this release.

Bugs fixed in version 0.74

  • Space allocated for exec envenvironment may be one byte short
  • Fix glob with patterns containing spaces, [ and ]
  • Sense of fconfigure -blocking is reversed
  • subst -noc should substitute vars inside brackets
  • Fix a reference counting bug
  • Fix invalid memory reference during finalisers
  • Fix an invalid memory reference in info references
  • Fix a buffer overflow in info references
  • Ensure that the full ref is passed to finalizer
  • Fix conversion of numbers >= 2^31
  • Fix binary scan for too-few bytes.
  • Fix bug that was causing the system environ variable to be freed
  • Fix a linenoise bug when moving off the left
  • Fix linenoise serial console window size for vt102
  • Fix a bug in the sqlite3 module (array index out of bounds)
  • Fix string replace replacing a single char

Features added in version 0.74

  • Add support for aio isatty
  • Remove octal literals with a leading 0 (http://www.tcl.tk/cgi-bin/tct/tip/114.html)
  • Add support for glob -directory
  • Allow abbreviations to glob options
  • Ensure that signals can break vwait
  • Allow aio copyto and aio read to copy >2GB
  • aio seek and aio tell should allow for 64 bit offsets
  • Add support for string equal -length and string compare -length

Performance Improvements

Thanks in large part to Lauri Kasanen, performance of Jim Tcl has improved by up to 20% in some areas compared to version 0.73.

    Executable Size     +1%
          PI digits     -4%
    [for] busy loop       .
       [local] loop       .
  [while] busy loop     -1%
                ary    -10%
         ary [dict]    -20%
       ary [static]     -1%
       dynamic code     -3%
dynamic code (list)       .
             expand     -1%
      fibonacci(25)     +1%
           heapsort     -4%
             mandel     -9%
         mini loops       .
       nested loops       .
             repeat       .
             rotate       .
              sieve    -20%
       sieve [dict]    -20%
              upvar    -16%
   wiki.tcl.tk/8566       .

Steve Bennett (steveb@workware.net.au)

Comments >>>

Coping without Namespaces - Revisited

Back in the article Coping without Namespaces, we discussed that fact that Jim Tcl did not support namespaces, but that small changes to the source could be made to allow namespace-based Tcl code to be ported for use with Jim Tcl.

Now with namespaces supported in Jim Tcl 0.73, porting Tcl code which makes use of namespaces is easier than ever.

Once again, let's consider porting dns.tcl from tcllib to Jim Tcl.


Firstly an explanation of what was changed.

--- dns.tcl.orig	2012-03-05 13:02:36.000000000 +1000
+++ dns.tcl	2012-03-05 13:02:56.000000000 +1000
@@ -1,3 +1,15 @@
+# dns.tcl - Steve Bennett <steveb@workware.net.au>
+#
+# Modified for Jim Tcl to:
+# - use udp transport by default
+# - use sendto/recvfrom
+# - don't try to determine local nameservers
+# - remove support for dns uris and finding local nameservers
+# - remove logging calls
+#   (both of these in order to remove dependencies on tcllib)
+
+# Based on:
+
 # dns.tcl - Copyright (C) 2002 Pat Thoyts <patthoyts@users.sourceforge.net>
 #
 # Provide a Tcl only Domain Name Service client. See RFC 1034 and RFC 1035

For simplicity, we remove the dependencies on tcllib logger, uri and ip to allow this example to be self contained. In Jim Tcl the binary and namespace modules are optional, so load them if required.

@@ -31,14 +43,11 @@
 #
 # $Id: dns.tcl,v 1.36 2008/11/22 12:28:54 mic42 Exp $
 
-package require Tcl 8.2;                # tcl minimum version
-package require logger;                 # tcllib 1.3
-package require uri;                    # tcllib 1.1
-package require uri::urn;               # tcllib 1.2
-package require ip;                     # tcllib 1.7
+package require binary
+package require namespace
 
 namespace eval ::dns {
-    variable version 1.3.3
+    variable version 1.3.3-jim2
     variable rcsid {$Id: dns.tcl,v 1.36 2008/11/22 12:28:54 mic42 Exp $}
 
     namespace export configure resolve name address cname \

Since Jim Tcl supports udp out-of-the-box, and it is more efficient, default to udp rather than tcp. Also comment out the logging calls.

@@ -49,23 +58,13 @@
         array set options {
             port       53
             timeout    30000
-            protocol   tcp
+            protocol   udp
             search     {}
             nameserver {localhost}
             loglevel   warn
         }
-        variable log [logger::init dns]
-        ${log}::setlevel $options(loglevel)
-    }
-
-    # We can use either ceptcl or tcludp for UDP support.
-    if {![catch {package require udp 1.0.4} msg]} { ;# tcludp 1.0.4+
-        # If TclUDP 1.0.4 or better is available, use it.
-        set options(protocol) udp
-    } else {
-        if {![catch {package require ceptcl} msg]} {
-            set options(protocol) udp
-        }
+        #variable log [logger::init dns]
+        #${log}::setlevel $options(loglevel)
     }
 
     variable types

udp is built-in with Jim Tcl.

@@ -248,14 +237,6 @@
         return -code error "no nameserver specified"
     }
 
-    if {$state(-protocol) == "udp"} {
-        if {[llength [package provide ceptcl]] == 0 \
-                && [llength [package provide udp]] == 0} {
-            return -code error "udp support is not available,\
-                get ceptcl or tcludp"
-        }
-    }
-    
     # Check for reverse lookups
     if {[regexp {^(?:\d{0,3}\.){3}\d{0,3}$} $state(query)]} {
         set addr [lreverse [split $state(query) .]]

udp in Jim Tcl works just like tcp, with readable event handler being triggered when the response is available.

@@ -273,6 +254,7 @@
         }
     } else {
         UdpTransmit $token
+        wait $token
     }
     
     return $token

Jim Tcl has no support for async connect, and the parameters to socket are a little different.

@@ -668,9 +650,9 @@
                                    "operation timed out"]]
     }
 
-    # Sometimes DNS servers drop TCP requests. So it's better to
-    # use asynchronous connect
-    set s [socket -async $state(-nameserver) $state(-port)]
+    # Jim Tcl has no async connect ...
+
+    set s [socket stream $state(-nameserver):$state(-port)]
     fileevent $s writable [list [namespace origin TcpConnected] $token $s]
     set state(sock) $s
     set state(status) connect

Comment out the async connect check.

@@ -683,11 +665,13 @@
     upvar 0 $token state
 
     fileevent $s writable {}
-    if {[catch {fconfigure $s -peername}]} {
-	# TCP connection failed
-        Finish $token "can't connect to server"
-	return
-    }
+
+    # Jim Tcl has no async connect ...
+#    if {[catch {fconfigure $s -peername}]} {
+#	# TCP connection failed
+#        Finish $token "can't connect to server"
+#	return
+#    }
 
     fconfigure $s -blocking 0 -translation binary -buffering none
 

udp in Jim Tcl is easy. Simpy create the socket with socket dgram and send with sendto.

@@ -722,18 +706,10 @@
                                   "operation timed out"]]
     }
     
-    if {[llength [package provide ceptcl]] > 0} {
-        # using ceptcl
-        set state(sock) [cep -type datagram $state(-nameserver) $state(-port)]
-        fconfigure $state(sock) -blocking 0
-    } else {
-        # using tcludp
-        set state(sock) [udp_open]
-        udp_conf $state(sock) $state(-nameserver) $state(-port)
-    }
-    fconfigure $state(sock) -translation binary -buffering none
+    set state(sock) [socket dgram]
+    #fconfigure $state(sock) -translation binary -buffering none
     set state(status) connect
-    puts -nonewline $state(sock) $state(request)
+    $state(sock) sendto $state(request) $state(-nameserver):$state(-port)
     
     fileevent $state(sock) readable [list [namespace current]::UdpEvent $token]
     

Reading from a udp socket is best done with recvfrom

@@ -879,7 +855,7 @@
     upvar 0 $token state
     set s $state(sock)
 
-    set payload [read $state(sock)]
+    set payload [$state(sock) recvfrom 1500]
     append state(reply) $payload
 
     binary scan $payload S id

Jim Tcl has lreverse built-in

@@ -1011,17 +987,6 @@
 }
 
 # -------------------------------------------------------------------------
-# Description:
-#   Reverse a list. Code from http://wiki.tcl.tk/tcl/43
-#
-proc ::dns::lreverse {lst} {
-    set res {}
-    set i [llength $lst]
-    while {$i} {lappend res [lindex $lst [incr i -1]]}
-    return $res
-}
-
-# -------------------------------------------------------------------------
 
 proc ::dns::KeyOf {arrayname value {default {}}} {
     upvar $arrayname array

Notice that no namespace-related changes were required when porting this module.

The latest version of dns.tcl for Jim Tcl is available in git.

Steve Bennett (steveb@workware.net.au)

Comments >>>

See All News Articles »