Using Cygwin and making it behave more like a Package Manager

Using the possibilities of the next to nothing alternative

Sometimes you’re stuck at a Windows box, and you long for all those trusty, handy Unix/Linux tools.
Then it is good to know that you can install Cygwin, as I’m sure most developers have heard of.

Simply put, it is a “near POSIX” standard “Unix like” environment for MS Windows® that let’s you install your Korn shell or ZSH for that matter, and pretend you’re using a decent computer operating system.

And, quoting the site:

  • a DLL (cygwin1.dll) which provides substantial POSIX API functionality.

Anyway. I haven’t been using it much, so this “tip” might turn up to be old news for many, but there will always be new beginners, right?

The point of this tip is to inform you that the file you download from Cygwin’s web site actually takes command-line arguments worth knowing about.

According to their site they claim that each time you want to add or remove program packages to the Cygwin installation, you need to download the install file anew and make the desired changes to your installation.

This is something that has annoyed me, after being used to installing pre-built software with a decent Package Manager like dpkg and it’s front-ends or RPM tools.

A Google Search away (“Duck Duck Go search”, Actually), I found a list of handy cmd-line options to use with the downloaded .exe setup file.

One thing should be mentioned though: The installer, as any other piece of software tends to change over time.
I suspect this is why Cygwin thinks it is best to always use the latest version of their installer, and being a software developer myself I do agree.

So, the first time you install Cygwin using the installer (this is the 64-bit version) make sure to install the “curl” and the “wget” tools. Then you can easily make a .bat file to download the latest version to the directory of choice before running it with the command line parameters described.

Here is a descriptive list over command line arguments to the .exe installer.
If you’re on a machine with no administration rights, you could use the –no-admin switch as an example.

Command Line Options:
 -D --download                     Download from internet
 -L --local-install                Install from local directory
 -s --site                         Download site
 -O --only-site                    Ignore all sites except for -s
 -R --root                         Root installation directory
 -x --remove-packages              Specify packages to uninstall
 -c --remove-categories            Specify categories to uninstall
 -P --packages                     Specify packages to install
 -C --categories                   Specify entire categories to install
 -p --proxy                        HTTP/FTP proxy (host:port)
 -a --arch                         architecture to install (x86_64 or x86)
 -q --quiet-mode                   Unattended setup mode
 -M --package-manager              Semi-attended chooser-only mode
 -B --no-admin                     Do not check for and enforce running as
                                   Administrator
 -h --help                         print help
 -l --local-package-dir            Local package directory
 -r --no-replaceonreboot           Disable replacing in-use files on next
                                   reboot.
 -X --no-verify                    Don't verify setup.ini signatures
 -n --no-shortcuts                 Disable creation of desktop and start menu
                                   shortcuts
 -N --no-startmenu                 Disable creation of start menu shortcut
 -d --no-desktop                   Disable creation of desktop shortcut
 -K --pubkey                       URL of extra public key file (gpg format)
 -S --sexpr-pubkey                 Extra public key in s-expr format
 -u --untrusted-keys               Use untrusted keys from last-extrakeys
 -U --keep-untrusted-keys          Use untrusted keys and retain all
 -g --upgrade-also                 also upgrade installed packages
 -o --delete-orphans               remove orphaned packages
 -A --disable-buggy-antivirus      Disable known or suspected buggy anti virus
                                   software packages during execution.

Use Cygwins abilities as a Package Manager with Cygcheck

In fact, the cygcheck.exe program accompanying every installation makes Cygwin behave more like systems such as dpkg (apt-get) or RPM (Yum / Zypper)

Some common tasks using cygcheck:

The -c option checks the version and status of installed Cygwin packages. If you specify one or more package names, cygcheck will limit its output to those packages, or with no 
arguments it lists all packages.

$ cygcheck -c
Cygwin Package Information
Package             Version         Status

alternatives        1.3.30c-10      OK
autoconf            13-1            OK
autoconf2.1         2.13-12         OK
autoconf2.5         2.69-2          OK
...

If you list one or more programs on the command line with -s, cygcheck will diagnose the 
runtime environment of that program or programs, providing the names of DLL files of which 
the program depends.

$ cygcheck -s vim
Found: D:\cygwin64\bin\vim.exe
Found: D:\cygwin64\bin\vim.exe
D:\cygwin64\bin\vim.exe
D:\cygwin64\bin\cygwin1.dll
C:\Windows\system32\KERNEL32.dll
C:\Windows\system32\api-ms-win-core-rtlsupport-l1-2-0.dll
C:\Windows\system32\ntdll.dll
C:\Windows\system32\KERNELBASE.dll
C:\Windows\system32\api-ms-win-core-apiquery-l1-1-0.dll
C:\Windows\system32\api-ms-win-core-processthreads-l1-1-2.dll
...

Searching all packages for a specific file:

$ cygcheck -p 'cygintl-2\.dll'

Found 1 matches for 'cygintl-2\.dll'.
libintl2-0.12.1-3 GNU Internationalization runtime library

$ cygcheck -p 'libexpat.*\.a'
Found 2 matches for 'libexpat.*\.a'.
expat-1.95.7-1 XML parser library written in C
expat-1.95.8-1 XML parser library written in C.

Here is a more complete list of alternatives:

At least one command option or a PROGRAM is required, as shown above.
-c, --check-setup show installed version of PACKAGE and verify integrity 
(or for all installed packages if none specified)
-d, --dump-only just list packages, do not verify (with -c)
-s, --sysinfo produce diagnostic system information (implies -c -d)
-r, --registry also scan registry for Cygwin settings (with -s)
-k, --keycheck perform a keyboard check session (must be run from a
 plain console only, not from a pty/rxvt/xterm)
-f, --find-package find the package to which FILE belongs
-l, --list-package list contents of PACKAGE (or all packages if none given)
-p, --package-query search for REGEXP in the entire cygwin.com package

Repository (requires internet connectivity)

Search for software:
Note: -c, -f, and -l only report on packages that are currently installed. To search all 
official Cygwin packages use -p instead. The -p REGEXP matches package names, descriptions, and names of files/paths within all packages.
--delete-orphaned-installation-keys

Delete installation keys of old, now unused
installations from the registry. Requires the right to change the registry.
--enable-unique-object-names Cygwin-DLL
--disable-unique-object-names Cygwin-DLL
--show-unique-object-names Cygwin-DLL

Enable, disable, or show the setting of the
\"unique object names\" setting in the Cygwin DLL
given as argument to this option. The DLL path must
be given as valid Windows(!) path.
See the users guide for more information.
If you don't know what this means, don't change it.

-v, --verbose produce more verbose output
-h, --help annotate output with explanatory comments when given
with another command, otherwise print this help
-V, --version print the version of cygcheck and exit
-c, --check-setup show installed version of PACKAGE and verify integrity 
(or for all installed packages if none specified)
-d, --dump-only just list packages, do not verify (with -c)
-s, --sysinfo produce diagnostic system information (implies -c -d)
-r, --registry also scan registry for Cygwin settings (with -s)
-k, --keycheck perform a keyboard check session (must be run from a
 plain console only, not from a pty/rxvt/xterm)
-f, --find-package find the package to which FILE belongs
-l, --list-package list contents of PACKAGE (or all packages if none given)
-p, --package-query search for REGEXP in the entire cygwin.com package

Repository (requires internet connectivity)

Search for software:
Note: -c, -f, and -l only report on packages that are currently installed. To search all 
official Cygwin packages use -p instead. The -p REGEXP matches package names, descriptions, and names of files/paths within all packages.

--delete-orphaned-installation-keys

Delete installation keys of old, now unused
installations from the registry. Requires the right to change the registry.
--enable-unique-object-names Cygwin-DLL
--disable-unique-object-names Cygwin-DLL
--show-unique-object-names Cygwin-DLL

Enable, disable, or show the setting of the
\"unique object names\" setting in the Cygwin DLL
given as argument to this option. The DLL path must
be given as valid Windows(!) path.
See the users guide for more information.
If you don't know what this means, don't change it.

-v, --verbose produce more verbose output
-h, --help annotate output with explanatory comments when given
with another command, otherwise print this help
-V, --version print the version of cygcheck and exit
-c, --check-setup show installed version of PACKAGE and verify integrity 
(or for all installed packages if none specified)
-d, --dump-only just list packages, do not verify (with -c)
-s, --sysinfo produce diagnostic system information (implies -c -d)
-r, --registry also scan registry for Cygwin settings (with -s)
-k, --keycheck perform a keyboard check session (must be run from a
 plain console only, not from a pty/rxvt/xterm)
-f, --find-package find the package to which FILE belongs
-l, --list-package list contents of PACKAGE (or all packages if none given)
-p, --package-query search for REGEXP in the entire cygwin.com package

Repository (requires internet connectivity)

Search for software:
Note: -c, -f, and -l only report on packages that are currently installed. To search all 
official Cygwin packages use -p instead. The -p REGEXP matches package names, descriptions, 
and names of files/paths within all packages.

--delete-orphaned-installation-keys

Delete installation keys of old, now unused
installations from the registry. 
Requires the right to change the registry.

--enable-unique-object-names Cygwin-DLL
--disable-unique-object-names Cygwin-DLL
--show-unique-object-names Cygwin-DLL

Enable, disable, or show the setting of the
\"unique object names\" setting in the Cygwin DLL
given as argument to this option. The DLL path must
be given as valid Windows(!) path.
See the users guide for more information.
If you don't know what this means, don't change it.

-v, --verbose produce more verbose output
-h, --help annotate output with explanatory comments when given
with another command, otherwise print this help
-V, --version print the version of cygcheck and exit

For those of you that want more in-depth information on using Cygwin for programming or just as a way to use Windows more like a decent OS, here is a direct link to the Cygwin PDF User Guide.

Until next time, happy Cygwin hacking…

Chris

I’m going away for 4-6 months, but please stay subscribed because…

New stuff is coming up!

Keywords: Game Development, Python, C, C++ and (hold your breath) Lisp!

New ways of thinking about the good old OOP, including functional development in Haskell and Test Driven Development and re-factoring legacy code in C++.

More 3D stuff, like Allegorithmic Substances and Modo 701.

The Blender Game Engine (programmed via Python) gets a thorough examination.

A taste of my upcoming book, called “Becoming a Real Life Developer” vol. 1 “The Stuff you Need-to-Know”.
This is a planned three-volume book series covering multiple programming languages, paradigms and languages plus how life unfolds in the real world of software development with version control systems, code reviews, the art of problem-solving and thinking “outside the box”.
It will also be geared towards team building and how to fit in to an established team of developers.
The art of becoming a software developer is very much a psychological challenge with a constant need to stay on top of the latest technologies.
My goal is to make one realize that this business is not always a bed of roses, but an extremely satisfying way of making a living if you know the right way to walk forward and get promoted without pushing yourself off the edge…

Until next time:

The Mad Man (going on a spiritual ride)

Using LINQ and EntityFramework, Measuring and analyzing the LINQ effectiveness

Some Entity Framework Components

When you use something like the extensive “Entity Framework” or any other ORM (Object Relational Mapping) tool, the process of writing sophisticated database driven applications becomes a lot easier.
It eliminates most of the data access code used to be required to create object-based data applications, and contains tools that makes it possible to create object-oriented data applications from an existing database, by using .edmx files that describes the CLR (Common Language Run-time) objects and these can be “extracted” from any existing relational database that the Entity Framework has a driver for.

This enables developers to program directly against data types defined at the conceptual level, presented as regular object instances.

But.
The responsibility to create clean queries and dispose the resources that’s being used, are just as important as it was in more “old fashioned” database connected applications that does all the SQL manually.

Many people assume that, as long as the application compiles and runs without complaining loudly, everything is probably done correctly, right?
This is .Net we’re talking about, so if something was wrong I would probably get a warning or something, right?

Sadly life is not that simple in this case.

The risk of introducing data-access “anti-patterns” in your code that may decrease the performance of your application and greatly slow down your organization’s database server is highly possible, and  actually quite likely to happen if you’re new to ORM and the Entity Framework in particular.

How Can You Measure the LINQ query’s Efficiency?

To reduce the chances of creating inefficient ORM based applications, we need some way to easily (or it won’t be used) analyze exactly what goes on in the EF brain when it executes our code.

This post focuses on a product called LINQ Insight by DevArt, that addresses this problem (and a lot of other LINQ related problems) and may be downloaded for free from the DevArt site:

Download LINQ Insight Standard

NB! The FULL functionality is free for 30 days, before it changes to the more restricted, but highly useful Free Edition.

Data access anti-patterns decreases performance greatly and must be avoided.

When you write applications using for example LINQ to Entities with the Entity Framework, what goes on behind the scenes is something like:

1) The Entity Framework “Mapped” Objects are created with a ObjectQuery() method, run against an existing ObjectContext instance.

The ObjectContext instance is a container for in-memory objects and metadata extracted from the relational database.
It manages the object identities and states, keeps a pointer to original and current values of the objects properties, and tracks changes made to each object in its cache store.

The ObjectContext provides the connection and metadata information that is required to compose and execute queries, and should therefore be set up in advance.

2) Entity Framework components like the ObjectQuery class are compiled behind the scenes, and generate “regular” SQL statements that does the “actual” query against the database.

The queried data is returned as entity objects, most often as anonymous class instances, and the application works with these objects instead of directly querying the SQL database.
This is an important point to understand.
Using LINQ in this way, to offload the database server is a fundamental aspect of ORM frameworks.

Constructing an ObjectQuery Instance

The ObjectQuery returns a collection of zero or more typed entities.  The ObjectQuery generic class implements the IQueryable generic interface, whose methods enable LINQ queries to be incrementally built. You can also let the compiler infer the type of entities by using the C# var keyword, which is something that simplifies specific query situations considerably.

Instances of the ObjectQuery generic class, which implements the generic IQueryable interface, serve as the data source for LINQ to Entities queries. In a query, you specify exactly the information that you want to retrieve from the data source. A query can also specify how that information should be sorted, grouped, and shaped before it is returned. In LINQ, a query is stored in a variable. This query variable takes no action and returns no data; it only stores the query information. After you create a query you must execute that query to retrieve any data.

// Call ctor with query for "persons" and the named ObjectContext
var personsQuery = new ObjectQuery("Persons", context);

3) You can get access to the entities via the reference and collection properties, and this in turn produces additional SQL queries to get the necessary data.

4) When your application finishes working out what entities are needed, the “mapping” process of EF is more or less done.

BUT: if there are any changes to the queried entities, the application will have to save them to database somehow.
It does this by transforming your LINQ, like SubmitChanges() to “regular” SQL statements, like SELECT, UPDATE etc.

All of this happens without you being able to “see” exactly what’s going on at a “lower” level. That’s kinda the idea, to abstract away the unnecessary details from the user, and let you think about designing the flow of the application, not the technicalities used to make things work.

Though all these actions are performed by Entity Framework run-time, and are transparent for the application that works with entities, they may be complex and affect application performance greatly. That’s why you probably need a good Entity Framework profiler to create efficient applications using Entity Framework for data access. Entity Framework profiler in LINQ Insight really profiles ORM events and allows you to see how much time LINQ statement or SubmitChanges call takes to execute as a whole even if it generated multiple SQL statements.

Example Application

Let’s show how to profile Entity Framework applications on a simple example. We have used the widely used NorthwindEF sample database for our example, and generated the model for it with EDM wizard using default settings. The application is a simple console application with the following code:

18 Reasons to Use C# / .Net

18 Reasons to Use C# / .Net

(some of this also explain why C# > Java)

This list is by no means complete, but may be a helper for those that wonder if they should jump on the old Java Wagon or the shiny C#/.Net Wagon with dual suspension and enhanced body construction…

  • No pointers required! C# programs typically have no need for direct pointer
    manipulation (BUT you are free to drop down to that level if absolutely
    necessary)
  • Automatic memory management through garbage collection. Given this, C# does
    not support a delete keyword.
  • Formal syntactic constructs for classes, interfaces, structures, enumerations, and
    delegates.
  • The C++-like ability to overload operators for a custom type, without the
    complexity (e.g., making sure to “return *this to allow chaining” is not your
    problem).
  • Support for attribute-based programming. This brand of development allows you
    to annotate types and their members to further qualify their behavior. For
    example, if you mark a method with the [Obsolete] attribute, programmers will
    see your custom warning message print out if they attempt to make use of the
    decorated member.
  • The ability to build generic types and generic members. Using generics, you are
    able to build very efficient and type-safe code that defines numerous placeholders
    specified at the time you interact with the generic item.
  • Support for anonymous methods, which allow you to supply an inline function
    anywhere a delegate type is required.
  • The ability to define a single type across multiple code files (or if necessary, as an
    in-memory representation) using the partial keyword.
  •  LINQ: Support for strongly typed queries used to interact with various forms
    of data.
  • Support for anonymous types that allow you to model the shape of a type rather
    than its behavior.
  • The ability to extend the functionality of an existing type (without subclassing)
    using extension methods.
  • Inclusion of a lambda operator (=>), which even further simplifies working with
    .NET delegate types.
  • An object initialization syntax, which allows you to set property values at the
    time of object creation.
  • Support for optional method parameters, as well as named method arguments.
  • Support for dynamic lookup of members at runtime via the dynamic keyword.
    This provides a unified approach to invoking members
    on the fly, regardless of which framework the member implemented (COM,
    IronRuby, IronPython, or via .NET reflection services)
  • Working with generic types is much more intuitive, given that you can easily map
    generic data to and from general System.Object collections via covariance and
    contravariance.
  • More and more companies and projects are going for the Open Source Mono Project’s implementation to create true cross-platform applications, both on desktop and mobile platforms.
  • The current version of C#
    provides a pair of new keywords (async and await), which massively simplify multithreaded and
    asynchronous programming.

You’ll be the judge! Both Languages has their charm, but personally (I learned Java many years ago) I strongly prefer C#, as you probably reasoned from this article…

I’m creating a new Blog these days!

While I’ll still maintain the thoughts of a Madman, I realize that its content starts to get a little bit fragmented. Therefore I’m creating a new Blog dedicated especially to C# and .Net, while also covering general application development topics and tools. The new creation lives over at Blogger at the following address:

“The C# Aficionado”

http://tcatitles.blogspot.com

If programming and C# is your thing, please consider following the Blog. 😉

Have a great weekend!

Some GNU Emacs hotkeys for ya!

I’ve collected some of the Emacs shortcuts / keyboard hotkeys I find myself using from time to time! 😉

Please visit:
GNU Emacs for Unix/Linux and precompiled Windows binaries
Aquamacs for OS X (I prefer this over http://emacsformacosx.com/ as it is better integrated)

To get a hold of this incredible “editor”. The reasons I write “editor” is because calling Emacs an editor is something of an understatement! It’s more like a little operating system or at least an intelligent Lisp world!

For you guys never bothered to learn Emacs: You don’t know what you’re missing out on!

Ok, here are the hotkeys (feel free to add some as a comment, and I’ll add them to the list!):

General commands:

c-x c-f = new buffer (file)
c-x c-c = quit
c-g = cancel input command
c-v next page
m-v previous page
c-l toggle cursor point up/down/center (pgUp/pgDown)

Navigation in text:

c-p = previous line
c-n = next line
c-b = backward
c-f = forward
m-f = move forward Word-by-word
c-a = beginning of the line
c-e = end of the line
m-a = beginning of the sentence
m-e = end of the sentence

m-< = beginning of the text
m-> = end of the text

Editing Text

Killing text:
(text goes to a safety buffer like in Cut)

m-backspace = kill word before cursor
m-d = kill word after cursor
m-k = kill words until end of sentence (.)
c-k = kill words until end of line
c-space = mark start of delete (move to end of text to delete)
c-w = delete text from mark until here
c-y = yank (paste) text from last kill buffer
m-y = cycle through the kill buffer entries

c-x u = undo
c-_ = undo too!

Search for text:
c-s = search forward in buffer from cursor
c-r = search backwards in buffer from cursor

for a non-interactive search press enter before search word.

Search and replace:
m-x replace-string + enter
<word to replace>
<word to replace with>

Managing buffers and windows

c-x c-b = display buffer list
c-x o = switch to buffer list, select with arrows and enter
c-x b = start typing the name of the buffer to choose (tab-complete)

c-x 1 = one buffer in window
c-x 2 = two buffers in window horizontal
c-x 3 = two buffers in window vertical

c-x 0 = close active buffer (file)

m-x recover-file = recover file after crash (open original file, not auto-backup)

c-u 90 c-x f = set margin width to 90 chars

c-u 0 c-l = move current line to top of buffer window
c-x o = move cursor to “other” buffer window
c-m-v = scroll through other window (keeping cursor in main window)

m-x make-frame = create an independent Emacs frame

esc esc esc = get out of recursive editing level mode

c-h f <function> = help on <function>
c-h k <command> = help on <command>
c-h a <keyword> = search for <keyword> in help
c-h i m Emacs = read Emacs manual using info

That is something to get you started!

Until next time, happy editing!

Chris

Pyrex

This is a promising project for those who need both speed (not the drug) and Python (not the .. no).

Very easy to combine C and Python with Pyrex, for those who didn’t know.

Check it out!

Pyrex.

A story about Lisp remote debugging

Revised slightly, but mostly taken from the free Apress book online: Practical Common Lisp

REPL = Read-Eval-Print Loop
SLIME = Superior Lisp Interaction Mode for Emacs

Even when the Lisp app is deployed, there’s often still a way to get to a REPL.
you can use the REPL and SLIME to interact with the Lisp that’s running a Web server at the same time as it’s serving up Web pages. It’s even possible to use SLIME to connect to a Lisp running on a different machine, allowing you for instance, to debug a remote server just like a local one.

An impressive instance of remote debugging occurred on NASA’s 1998 Deep Space 1 mission. A half year after the space craft launched, a bit of Lisp code was going to control the spacecraft for two days while conducting a sequence of experiments. Unfortunately, a subtle race condition in the code had escaped detection during ground testing and was already in space. When the bug manifested in the wild–100 million miles away from Earth–the team was able to diagnose and fix the running code, allowing the experiments to complete.

One of the programmers described it as follows:

Debugging a program running on a $100M piece of hardware that is 100 million miles away is an interesting experience. Having a read-eval-print loop running on the spacecraft proved invaluable in finding and fixing the problem.